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Preface 


As  technology  advances,  more  and  more  computer  systems  are  being 
interconnected  by  communication  networks  so  users  can  share  data  and 
applications.  Data  analysis  on  a  network  environment  containing 
multiple  computers  is  very  difficult.  The  global  time  transformation  ap¬ 
proach  reduces  the  difficulty  in  doing  the  analysis. 

This  Global  Time  Transformations  report  is  the  product  of  an  in- 
house  research  effort  in  the  area  of  distributed-parallel  computing.  The 
effort  involves  a  set  of  experiments  that  were  executed  over  a  distributed 
network  between  Rome  Laboratory  (RL)  in  Rome,  New  York  and  the 
Northeast  Parallel  Architecture  Center  (NPAC)  at  Syracuse  University  in 
Sjrracuse,  New  York.  During  each  of  the  experiments,  data  was  collected 
from  four  separate  sources.  To  do  an  analysis  of  the  experiment,  we  had 
to  reconstruct  the  order  of  the  events  from  the  data.  When  we  tried  to 
bring  the  separate  sets  of  data  together  to  reconstruct  the  events  during 
the  experiment,  we  encountered  a  problem.  Because  the  timestamp  data 
for  each  system  was  different,  there  was  no  way  to  relate  the  data  from 
one  system  to  other  system.  To  bring  the  data  together,  a  time  trans¬ 
formation  was  derived  that  would  map  each  system’s  data  to  a  common 
time  basefi.e.  the  global  time  base).  The  purpose  of  the  Global  Time 
Transformation  is  to  take  the  local  time  data  and  convert  it  to  the 
equivalent  global  time  data. 

Determining  the  exact  time  for  any  event  is  physically  impossible. 
The  intention  of  the  Global  Time  Transformation  is  to  map  the  timestamp 
datum  into  datum  that  is  much  closer  to  the  actual  global  time.  The 
accuracy  of  the  transformation  will  depend  on  the  precision  of  your 
global  time  transformation.  The  major  factors  that  limit  the  precision  are 
the  resolution  of  the  time  measurement  capability  and  observability  of 
the  metrics. 

This  report  describes  the  Global  Time  Tfansformations  part  of  the 
distributed-parallel  in-house  effort.  The  report  is  divided  up  into  seven 
sections.  Section  1  is  the  introduction  to  the  report  and  explains  where 
the  global  time  transformation  techniques  fit  into  the  general  technology 
base.  Section  2  defines  the  function  of  the  global  time  transformations 
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and  derives  the  transformation.  Section  3  defines  the  clock  drift  rate 
and  discusses  the  various  techniques  used  to  calculate  the  clock  rate. 
Section  4  defines  the  clock  offset  and  discusses  a  couple  of  the 
techniques  for  measuring  and  calculating  the  clock  offset.  Section  5 
compares  the  global  time  transformation  predictions  with  actual  mea¬ 
surements  of  clock  drift.  Section  6  recommends  candidate  areas  of  fol¬ 
low-on  research.  Section  7  is  the  conclusion  for  the  report. 
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Section  1:  Multiple  Clock  Systems 


Introduction 

As  computers  become  more  prevalent  and  applications  become 
more  complex,  it  is  not  surprising  that  many  solutions  to  today's  prob¬ 
lems  involve  multiple  computers.  As  technologies  like  distributed  pro¬ 
cessing  and  cluster  processing  mature,  there  will  be  a  significant  in¬ 
crease  in  the  number  of  multiple  computer  systems.  Unfortunately, 
when  new  technologies  provide  opportunities  to  solve  problems,  new 
problems  are  introduced.  One  of  those  problems  is  that  a  multiple  com¬ 
puter  system  has  multiple  clocks  and  each  clock  has  a  different  time  (see 
Figure  lA).  None  of  these  clocks  are  set  to  the  same  time  nor  do  they  run 
at  the  same  speed.  An  analysis  under  these  conditions  is  impossible.  If 
the  metric  for  your  analysis  is  time,  then  analyzing  a  multiple  clock  sys¬ 
tem  is  a  problem.  The  fundamental  problem  is  that  there  is  no  direct 
way  to  relate  the  data  from  one  system  to  the  data  from  another  system. 


Figure  lA 

There  are  many  multiple  clock  systems  and  situations  in  which 
data  from  multiple  clock  systems  have  to  be  used  or  analyzed.  One  sit- 
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uation  is  when  you  ti,  to  create  order  from  the  data^  For  example,  sup¬ 
pose  you  wanted  to  observe  the  execution  of  an  application  for  analytical 
reasons.  The  analyst  runs  an  experiment  and  collects  several  sets  of 
data.  Without  a  relationship  to  associate  the  data  in  one  time  domain  to 
the  data  in  the  other  time  domain,  there  is  no  way  to  order  the  events. 
Any  attempts  to  produce  the  proper  sequence  of  events  will  fail.  During 
the  ordering  process,  there  are  usually  some  logical  sequences  of  events 
that  can  be  used  to  detect  the  need  for  a  transformation.  For  example, 
the  arrival  time  for  a  message  on  one  system  can  precede  the  departure 
time  for  a  message  on  the  other  system,  'fhe  ordering  becomes  more  dif¬ 
ficult  when  the  observer  only  sees  a  small  part  of  an  event.  For  example, 
the  Rube  Goldberg  machine  is  a  machine  that  has  a  well  defined  se¬ 
quence  of  events  whose  goal  is  to  capture  a  mouse  or  turn  on  an  appli¬ 
ance.  Let  two  events  in  a  Goldberg’s  machine  sequence  be  a  ball  drop¬ 
ping  down  a  chute  and  the  ball  releasing  a  balloon.  Suppose  there  are 
two  observers  and  each  is  assigned  a  different  event  to  watch  and  record 
the  starting  time  for  the  event.  One  observer  saw  the  ball  being  dropped 
down  the  chute  at  12:00  PM  his  time  and  another  observer  saw  the  ball 
release  a  balloon  at  11:39  AM  his  time.  How  much  time  was  there  be¬ 
tween  the  time  the  ball  was  dropped  until  it  released  the  balloon?  The 
time  data  corresponding  to  the  events  can  not  be  used  to  answer  this 
question  without  some  way  of  relating  the  data  from  one  observer  to  the 
data  of  the  other. 

Merging  is  another  situation  where  you  encounter  the  multiple  clock 
problem.  The  motivation  for  merging  is  to  make  one  database  out  of 
multiple  data  bases  or  combine  subsets  of  multiple  data  bases  to  form  a 
new  data  base.  When  the  rules  for  merging  are  based  on  their  time  val¬ 
ues,  then  the  problem  surfaces  again.  If  data  values  in  one  database 
cannot  be  associated  with  the  data  in  the  other,  merging  of  the  databases 
is  not  possible.  Once  again,  some  relationship  is  required  to  associate 
the  data  in  one  system  to  the  data  in  the  other  system. 

Numerical  analysis  is  another  t)q)e  of  data  processing  that  encoun¬ 
ters  the  multiple  clock  problem.  Numerical  analysis  quite  often  uses 
data  from  multiple  clocks  to  do  calculations.  When  you  do  a  perfor- 

1  Since  the  events  can  occur  concurrently,  the  ordering  really  is  a  partial  ordering  of  the 
events. 
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mance  analysis  on  a  multiple  computer  system  or  distributed  system, 
data  for  the  analysis  come  from  different  computers  and  network  analyz¬ 
ers.  With  data  fn  its'  original  form,  it  is  impossible  to  use.  To  do  the 
analytical  calculations,  the  analysts  must  either  transform  the  data  from 
one  clock  into  the  time  base  of  the  other  or  transform  the  data  from  both 
clocks  to  a  common  time  base.  In  either  case,  there  is  a  need  to  trans¬ 
form  the  data. 

The  multiple  clock  problem  is  a  very  common  problem  and  a  solu¬ 
tion  needs  to  be  found  before  an  analysis  can  be  performed.  There  is  no 
exact  solution  to  the  multiple  clock  data  problem.  The  physics  of  the  sys¬ 
tem  (i.e.  software  and  hardware  delays  and  the  speed  of  an  electron  is 
finite /bounded)  make  it  impossible  to  measure  the  exact  time,  even  if 
there  is  only  one  clock.  The  measurements  will  always  be  approximate 
values  to  the  exact/actual  time. 

Hardware  Techniauea 

The  solutions  to  the  problem  are  either  to  set  and  synchronize  all 
the  clocks  so  that  they  will  always  read  the  same  or  to  accept  the  fact 
that  they  are  not  the  same  and  make  the  proper  adjustments  to  the  data. 
Of  those  techniques  that  attempt  to  synchronize  the  clocks,  the  most 
common  techniques  are  the  hardwire  connection,  the  radio  clock  and  the 
time  protocols  techniques.  The  most  common  technique  for  adjusting 
the  data  is  the  software  data  transformation  technique. 

Of  the  s)mchronization  techniques,  the  easiest  is  the  hardwire  con¬ 
nection.  There  are  two  hardwire  techniques.  The  first  hardwire  tech¬ 
nique  is  to  hardwire  all  the  timestamping  mechanisms  to  the  same  clock 
This  would  enable  the  separate  computer  systems  to  use  a  common 
clock.  There  are  implementation  problems,  bke  clock  contention,  which 
creates  wait  periods  that  impact  the  timestamp  data.  There  are  other 
problems  in  hardwiring  the  system.  Most  systems  cannot  be  hardwire 
modified  because  they  are  proprietary  and  the  schematics  are  not  avail¬ 
able.  Chip  technology  and  firmware  adds  to  the  difficulty  in  modifying 
these  systems.  Another  disadvantage  is  that  most  systems  usually  in¬ 
volve  a  distributed  network,  which  makes  hardwiring  not  feasible  or 
practical. 
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Another  hardwire  technique  uses  a  common  hardwire  to  synchronize 
the  multiple  clocks  to  read  the  same.  Once  again,  the  reasons  mentioned 
in  the  previous  technique  are  also  true  for  this  technique.  The  advantage 
is  that  there  is  less  clock  contention.  Hardwire  techniques  are  difficult 
and  sometimes  impossible  to  implement,  are  very  expensive  and  always 
contain  additional  hardware  delays. 

The  biggest  problem  with  hardwire  connection  techniques  is  that 
they  use  wire  for  connectivity.  The  hardwire  connection  technique 
becomes  less  feasible  and  less  cost  effective  when  the  distance  gets 
greater  then  a  few  thousand  feet.  One  way  to  eliminate  the  wire  is  to  use 
a  "radio  signal".  This  radio  signal  technique  broadcasts  a  signal  from  one 
location  and  each  of  the  other  systems  receives  the  radio  signal.  The 
signal  is  then  used  to  align  its  clock.  This  technique  is  similar  to  the  US 
radio  station  WWV,  in  Boulder  Colorado,  that  broadcasts  a  time-hack  ev¬ 
ery  minute  so  that  everybody  can  set  their  watches  and  clocks.  The  ad¬ 
vantage  of  this  technique  is  tha'  it  doesn’t  need  miles  of  wire  to  work. 
The  disadvantages  are  that  it  is  victim  to  propagation  delays,  it  requires 
expensive  hardware  (i.e.  radio  equipment  and  interface  boards)  and  the 
clock  circuitry  is  not  always  accessible  to  do  the  synchronized. 

One  of  the  problems  with  the  radio  signal  technique  is  the  expensive 
hardware.  The  logical  thing  is  to  reduce  or  eliminate  the  expensive 
hardware.  Let's  recall  that  the  operational  environment  for  this  effort  is  a 
set  of  computers  connected  by  a  communication  network.  Since  the 
connectivity  is  already  being  provided  by  the  network,  the  network  can  be 
used  to  do  the  synchronization.  To  take  advantage  of  the  network  con¬ 
nectivity.  special  timing  protocols  were  designed  specifically  for  S3nn- 
chronizing  clocks  (e.g.  the  Network  Time  Protocol  (NTP)  and  the  Digital 
Time  Service  (DTS))  [Mills  89b].  NTP  is  a  hierarchical  structure  of  pri¬ 
mary  and  secondary  servers  that  work  in  a  client/server  arrangement. 
The  client  system  dynamically  interacts  with  the  NTP  servers  to  S3mchro- 
nize  the  clients  clock.  DTS  is  a  synchronization  subnet  which  "...consists 
of  clerks,  servers,  couriers  and  time  providers.  With  respect  to  the  NTP 
nomenclature,  a  time  provider  is  a  primary  reference  source,  courier  is  a 
secondary  server  intended  to  import  time  from  one  or  more  distant  pri¬ 
mary  servers  for  local  redistribution,  and  a  server  is  intended  to  provide 
time  for  possibly  many  end  nodes  or  clerks."  [Mill89a.  Mill89b.  Mill92] 
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The  time  protocol  is  less  expensive  than  the  hardwire  techniques  but 
does  have  some  basic  requirements  that  must  be  met  by  the  client  sys¬ 
tem.  The  client  system  must  be  able  to  modify  the  clock,  create  and 
modify  messages  and  interact  with  the  servers.  There  are  two  major 
problems  with  the  timing  network  protocol  approach.  One,  many  sys¬ 
tems  can't  meet  all  the  functional  requirements.  Two,  there  is  still  some 
error  in  the  clock  due  to  the  delays  in  network  communication  and  pro¬ 
cessing  of  the  protocols.  In  the  recommended  research  section  of  this  re¬ 
port,  suggestions  are  made  as  to  a  possible  technique  for  reducing  the  er¬ 
ror  in  the  network  timing  protocol  techniques. 

Software  Techniques 

As  discussed  earlier,  sometimes  the  operational  environment  makes 
it  impossible  to  synchronize  the  clocks  and  that  alternative  solutions 
have  to  be  found.  One  class  of  solutions  assumes  that  multiple  unsy- 
chronized  clocks  generate  the  data  and  that  the  data  needs  to  be  ad¬ 
justed.  'fhe  adjustment  to  the  data  is  done  by  a  data  transformation. 
The  data  transformation  technique  maps  the  data  from  one  tim*"  domain 
to  a  common  time  domain  so  that  the  data  appears  to  have  con^e  from  a 
common  clock.  These  techniques  are  the  software  data  transformation 
techniques  and  are  categorized  as  an  on-the-fly  technique  or  an  after- 
the-fact  technique  or  a  hybrid  technique. 


Measurements  I  Analysis 


Time  Time  ^  Xime 

On-the-Fly  Technique 
Figure  IB 

The  on-the-fly  technique  takes  the  timestamp  d^tum  and  trans¬ 
forms  it  to  a  global  time  datum  where  it  is  either  used  immediately  or  is 
stored  to  be  used  later  (r.ee  Figure  IB).  One  advantage  of  the  on-the-fly 
technique  is  that  the  data  appears  in  only  one  form  which  requires  less 
storage.  Another  advantage  is  that  the  data  is  transformed  only  once 
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and  the  analysis  program  doesn't  have  to  spend  time  doing  the  trans¬ 
form.  A  disadvantage  of  the  technique  is  that  the  original  data  is  not 
stored.  If  something  happens  during  the  transformation,  the  data  is  lost. 
Another  disadvantage  is  that  implementing  this  technique  on  a  unipro¬ 
cessor  system  results  in  a  shift  in  the  data  measurements.  If  a  unipro¬ 
cessor  is  doing  the  transformation  on  previously  measured  data  when  it 
is  time  for  a  new  measurement,  the  measurement  is  delayed.  This  delay 
produces  an  error  in  the  measurements.  The  magnitude  of  the  shift 
varies  in  time  and  depends  on  the  current  environment.  This  could  be 
reduced/eliminated  in  a  multi-tasking  environment  by  having  the  mea¬ 
surement  processing  independent  of  the  normal  processing. 


Ifeasurement*  |  Analysis 
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After-the-Fact  Technique 
Figure  1C 


For  the  cjter-the-fact  technique  the  data  is  taken,  stored  and  later 
transformed  to  global  time  data  and  used  for  the  calculations  (see  Figure 
1C).  This  technique  has  an  advantage  that  it  preserves  the  original  data. 
This  is  important  if  the  data  is  to  be  used  for  multiple  applications  or  if  a 
problem  occurs  in  during  the  transformation.  Another  advantage  of  the 
technique  is  it  delays  the  transformation  until  later,  so  the  data  mea 
surements  are  more  accurate  because  there  is  no  contention  for  the 
uniprocessor  when  the  measurements  are  being  taken.  A  disadvantage 
is  that  it  requires  more  storage  then  an  on-the-fly  technique  because  the 
data  exists  in  two  forms,  the  original  and  transformed  form.  If  the  same 
data  is  accessed  often  it  would  be  advantageous  to  process  and  store  the 
data  in  the  translated  forms.  This  is  done  by  transforming  either  the  en¬ 
tire  set  of  data  or  as  a  subset  of  the  data.  Another  disadvantage  is  that 
the  transformation,  when  done  during  the  analysis,  slows  down  the  anal¬ 
ysis  because  it  uses  some  of  the  processor  time.  This  could  be  a  problem 
for  those  analyses  which  are  data  intensive. 
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The  best  technique  is  the  hybrid  technique  which  gives  the  analyst 
the  best  of  both  techniques,  even  on  a  uniprocessor  system  (see  Figure 
ID).  The  basic  difference  is  that  this  technique  takes  three  steps  to 
complete  while  the  previous  techniques  use  only  two  steps.  Step  one,  the 
data  is  measured  and  stored  during  the  measurements  part  of  the 
experiment  like  the  after-the-fact  technique.  Therefore,  the  measured 
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Hybrid  Technique 
Figure  ID 


data  is  not  effected  because  the  processor  is  dedicated  to  the  measure¬ 
ment  process.  Step  two,  after  all  the  measured  data  has  been  taken,  the 
time  transformation  maps  the  data  into  the  common  time  value  which  is 
stored  into  permanent  storage.  Step  three,  the  transformed  data  is  then 
used  to  do  the  data  analysis. 

The  additional  step  is  step  two  where  the  technique  preprocesses  the 
data  before  the  analysis.  One  advantage  is  that  the  measured  data  is 
more  precise  because  the  data  is  not  affected  by  the  delay  generated  by 
the  transformation  processing.  Another  advantage  is  that  the  data  are 
stored  in  both  the  original  and  transformed  forms.  The  technique  is 
more  reliable  because  the  error  recovery  possibilities  are  better.  Another 
benefit  is  that  during  the  data  analysis  there  is  no  processing  time  re¬ 
quired  to  transform  the  data.  This  time  savings  speeds  up  the  analysis. 
EXren  more  time  is  saved  when  multiple  requests  are  made  for  the  same 
data  entry.  The  main  disadvantage  is  that  the  data  must  be  stored  in 
both  forms  at  the  same  time.  If  the  set  of  data  is  large,  then  the  final 
storage  requirement  is  very  large. 

This  section  discussed  some  of  the  primary  solutions  to  the  multiple 
clock  data  problem  along  with  the  advantages  and  disadvantages.  From 
the  list  of  candidate  solutions,  the  software  solutions  are  the  most  feasi¬ 
ble,  flexible,  and  cost  effective.  A  close  look  at  the  software  techniques 
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The  Perfect  Clock 

The  ideal  situation  for  data  analysis  is  when  the  time  data  from  each 
clock  in  a  multiclock  system  always  reads  exactly  the  same  as  every  other 
clock  in  the  system.  For  all  these  clocks  to  always  read  the  same,  they 
must  all  be  set  to  the  same  time,  must  always  run  at  the  same  speed  and 
the  speed  must  be  constant.  Clocks  that  can  do  this  are  "perfect  clocks." 

A  formal  definition  of  a  perfect  clock  is  a  clock  that  is  correct,  accu¬ 
rate.  and  stable  [Duda  87].  A  clock  is  "correct'  if  the  reading  on  the  clock 
is  the  same  as  the  reference  time.  A  clock  is  "accurate"  if  the  speed  of 
the  clock  is  the  same  as  the  speed  of  the  reference  clock.  This  means  that 
a  second  on  one  clock  is  the  same  as  a  second  on  the  other  clock.  The 
speed  of  the  clock  can  be  determined  by  examining  the  derivative  of  the 
clock  readings  with  respect  to  time.  At  those  times  where  the  derivative 
is  1  the  clock  is  accurate^  .  A  clock  is  "stable"  if  the  difference  in  the 
speeds  between  the  clock  and  the  reference  clock  is  a  zero.  The  stability 
of  the  clock  can  be  determined  by  examining  the  derivative  of  the  clock 
speed  with  respect  to  time.  At  those  times  where  the  difference  is  0.  the 
clock  is  stable.  For  a  clock  to  be  a  "perfect"  clock  it  must  always  read  the 
same  time  as  the  reference  clock,  must  always  change  time  at  the  same 
rate  as  the  reference  clock  and  the  clock  rate  never  changes. 

The  global  time  transformation  is  a  transformation  which  maps  local 
clock  times  into  global  (i.e.  reference)  clock  times  (see  figure  2A).  The 
global  time  transformation  expressed  as  a  polynomial  of  infinite  order 
would  look  like 

G(ti^Qcal)  ~  3-0  ^  f  Local  ^  *  ttocal  ^  Si3  *  tLocal  Si*  *  ttocal  SLs  *  tuocal"^**" 


^  Note:  A  clock  being  accurate  does  not  mean  that  the  clock  reading  and  the  reference  time 
are  the  same.  For  both  clocks  to  have  the  same  reading  the  clocks  have  to  be  correct. 
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For  a  perfect  clock,  the  global  time  transformation  would  be 

Local )  =  0  +  (1)  •  tLocal  ■*"  *  tLocal 

~  ^Local 

and  the  higher  order  coefficients  are  0.  A  clock  is  perfect  when  the  local 
timefi.e.  tLocal)  is  equal  to  the  global  time  (i.e.  G(tLocai)  =  tLocal)-  Unfortu¬ 
nately,  the  clocks  that  are  in  the  multiple  clock  systems  are  not  perfect 
clocks.  So,  the  coefficients  of  the  polynomial  expression  for  real  clocks 
are  ao^^O,  ai  ^  1  and  and  the  higher  order  coefficients  are  not  always 
0. 


GLOBAL  TIME  TRANSFORMATION 


General  Global  Time  Transformation 
Figure  2A 


The  perfect  clock  definition  and  transformation  gives  some  insight 
into  what  are  the  key  characteristics  of  clocks.  The  clock  closest  to  being 
a  perfect  clock  is  the  atomic  clock.  The  atomic  clock  still  has  to  have  pe¬ 
riodic  adjustments  made  to  it  to  compensate  for  the  fact  that  it  is  not  a 
perfect  clock.  Since  the  system's  clock  is  not  a  perfect  clock,  it  drifts 
relative  to  the  global  clock  and  obviously  requires  adjusting  to  keep  it 
close  to  being  correct,  accurate  and  stable. 

AT3rpiyftlSyytgm 

The  next  step  is  to  find  out  how  perfect  is  a  typical  system  clock.  To 
examine  the  clock  behavior,  we  implemented  a  set  of  experiments  which 
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enabled  us  to  observe  some  electronic  clocks,  record  their  times  and  ana¬ 
lyze  their  time  data.  Figure  2B  is  a  diagram  of  the  hardware  configura¬ 
tion  for  the  experiment.  Clock  G  is  the  reference  clock  and  will  be  the 
clock  that  all  other  times  will  be  compared  with  and  mapped  for  this 
report.  The  software  controlled  the  experiment  and  the  message  timing, 
recorded  and  stored  the  timestamp  data  and  did  the  analysis. 


Configuration  of  the  Experiment 
Figure  2B 


In  the  experiment  that  was  implemented,  clocks  X  and  Y  were  LAN  proto¬ 
col  anal5rzers  each  having  an  internal  clock.  Systems  Z  and  G  are  parallel 
computers.  The  experiment  was  simple.  At  hour  intervals,  systems  X,  Y 
and  Z  send  a  message  to  system  G.  System  G  receives,  timestamps  and 
stores  the  data  .  After  completing  the  experiments,  the  observed  differ¬ 
ences  between  clocks  are  calculated.  The  results  of  the  calculation  are 
shown  in  Figures  2Ca,  2Cb  and  2Cc.  The  plot  is  a  locus  of  boxes,  t. 
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Figure  2Cb 
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which  mark  the  data  points  and  a  linear  approximation  to  the  data 
points. 

All  three  of  these  plots  have  three  fundamental  characteristics. 

1)  each  clock  was  not  initially  set  to  the  same  time  as  clock  G. 

2)  each  clock  drifts  at  a  different  rate  and 

3)  each  plot  appears  to  be  linear. 

To  develop  an  understanding  for  the  characteristics  of  the  data,  we  curve 
fit  each  of  the  data  sets  and  looked  at  the  coefficients.  The  results  of  the 
curve  fitting  are  in  the  following  table: 


Clock  X 

Clock  Y 

Clock  Z 

8.406 

37.8 

19.45 

0.0799 

0.0787 

0.0748 

^2 

0.000059948 

0.000059946 

0.000077183 

^3 

0.000000567 

0.000000569 

0.000000220 

a4 

0.0000000012 

0.0000000012 

0.0000000024 

Table  A 


Notice  that  the  coefficients  for  the  higher  order  terms  are  quickly  ap¬ 
proaching  zero.  This  was  expected  because  the  plot  looked  linear.  Obvi¬ 
ously,  a  good  approximation  for  the  curve  should  be  a  linear  equationfi.e. 
G(tLocai)  =  ao  +  ai*tLocai  )•  To  Confirm  this,  let’s  look  at  how  long  it  would 
take  before  the  contribution  of  each  of  the  terms  of  the  pol5niomial  would 
become  measurable.  Let's  assume  that  the  resolution  of  the  timestamp 
mechanism  is  1  millisecond.  Table  B  shows  how  much  time  has  to  elapse 
before  the  contribution  of  a  term  would  be  big  enough  to  be  measurable. 

There  is  quite  a  difference  in  time  between  the  ai  term  and  the  a2 
term.  The  time  to  contribute  1  millisecond  goes  from  45  seconds  to  over  4 
hours  on  clock  X.  Clock  Y  and  Clock  Z  have  the  same  characteristics. 

Curve  fitting  techniques  attempt  to  derive  a  polynomial  that  passes 
through  each  of  the  data  points.  The  problem  with  the  curve  fitting 
technique  is  the  derived  polynomial  function  is  not  very  accurate  between 
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the  data  points  when  the  number  of  data  points  gets  greater  then  10.  For 
those  cases  where  the  measured  data  is  polynomial  in  nature,  the  interpo¬ 
lation  of  the  values  of  the  function  between  the  data  points  is  good.  Not 
all  phenomenon  is  of  a  pol)momial  nature  and  for  those  cases  the  interpo- 


Clock  X 

Clock  Y 

Clock  Z 

^0 

0.0000 

0.0000 

0.0000 

ai 

.0125 

.0127 

.0134 

^2 

4.0843 

4.0843 

3.5995 

as 

12.0820 

12.0678 

16.5650 

30.2100 

30.1200 

25.4070 

Number  of  hours  before  the  term  contributes  1  Millisecond 

Table  B 


lation  is  poor.  The  accuracy  of  the  interpolation  depends  on  similarity  be¬ 
tween  the  curve  fitted  polynomial  and  the  phenomenon  that  is  being  mea¬ 
sured. 

Each  of  the  above  graphs  has  over  200  data  points.  In  general,  the 
derived  polynomial,  that  passes  through  this  many  points,  would  have 
higher  coefficients  with  larger  values.  For  the  higher  order  coefficients  to 
be  so  small  with  these  many  data  points  means  that  the  function  is 
dominated  by  the  lower  order  coefficients.  Since  there  is  only  zero  order 
and  first  order  terms,  the  function  is  linear. 

The  Global  Time  Transformation 

From  Table  A  it  is  obvious  that  the  dominant  terms  for  all  three  systems 
are  the  zero  and  first  order  terms.  The  polynomial  that  best  fits  this  locus 
of  points  is 

G(t(^)  =  ao  +  ai 

*  f  Local  (1) 

which  is  linear.  The  global  time  transformation  should  be  a  function  that 
has  the  same  form  as  Equation  (1).  The  two  coefficients  of  the  global  time 
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transformation,  G(tLocai).  are  the  only  two  values  that  have  to  be  deter¬ 
mined  for  the  global  time  transformation.  Figure  2D  is  a  plot  of  a  typical 
set  of  timestamp  data.  The  ao  coefficient  is  the  clock  offset  which  is  the 
global  time  when  the  local  time  is  zero.  This  coefficient  is  zero  only  when 


The  global  time  transformation  Model 
Figure  2D 

the  clocks  are  initially  set  to  the  exact  time.  For  the  global  time  transfor¬ 
mation  to  be  correct  for  all  time,  this  term  has  to  be  zero. 

The  ai  coefficient  is  the  clock  rate.  The  clock  rate  is  the  rate  at  which 
the  clock  is  changing  time.  The  terms  clock  rate  and  clock  drift  rate  are 
often  used  interchangeably.  The  two  terms  are  not  the  same  but  are  re¬ 
lated.  The  difference  will  be  discussed  in  detail  in  Section  3.  The  ai  coef¬ 
ficient  in  Equation  (1)  deals  with  the  speed  of  the  clock.  Because  it  is  not 
possible  to  build  with  today's  technology  two  clocks  that  change  time 
exactly  at  the  same  speed,  the  ai  coefficient  is  never  1. 

In  the  Equation  (1)  of  the  global  time  transformation,  the  *  operation 
in  the  term  ai*tLocai.  has  to  be  clarified.  The  coefficient  ai  is  a  number 
that  is  base  10  and  ttocai  is  a  number  that  is  base  hour-second-minute. 
Since  the  operation  •  is  not  defined  for  the  operands  of  different  bases,  a 
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conversion  has  to  be  done  on  one  of  the  operands.  The  ttocai  operand  has 
to  be  converted  to  a  base  10  and  the  operation  •  for  base  10  is  used  or  the 
ai  has  to  be  converted  to  an  hour-second-minute  number  and  the  opera¬ 
tion  •  for  hour-second-minutes  be  used.  Since  the  base  10  operations  are 
easier  than  hour-second-minutes  operations  to  implement,  the  conver¬ 
sion  will  be  to  base  10. 

Since  the  prediction  of  the  global  time  transformation  is  base  hour- 
second-minutes,  the  global  time  transformation  must  convert  timestamp 
data  {base  hour-second-minute)  to  elapse  time  (base  10),  then  do  the 
global  transformation  and  then  invert  the  transformation  back  to  clock 
timeCbase  hour-second -minute). 

In  summary,  the  global  time  transformation  takes  the  clock  reading 
from  any  clock  in  the  multiclock  system  and  calculates  the  corresponding 
clock  reading  for  the  global  clock. 

The  only  information  that  is  needed  to  use  the  Global  Time  Trans¬ 
formation  is  the  clock  offset  and  the  clock  rate  for  the  system’s  clock.  A 
discussion  of  the  clock  rate  and  the  clock  offset  is  given  in  the  Sections  3 
and  4  respectively. 
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Deflnitiona 

This  section  derives  the  equations  for  calculating  the  clock  rate. 
Formal  definitions  are  given  for  the  terms  and  notation  that  is  used  in  the 
derivation  of  the  clock  rate  equations.  As  a  prelude  to  the  derivation, 
background  information  is  given  on  the  clocking  mechanisms,  types  of 
messages  and  measurement  techniques.  This  information  enables  the 
reader  to  follow  the  derivation  of  the  clock  rate  equations. 

The  Global  time  transformation  is  a  pol3niomial  function  (i.e.  Gftiocai) 
=  ao  +  ai*tLocai)  which  contains  only  two  coefi&cients.  In  this  section,  the 
coefficient  ai,  the  clock  rate  will  be  discussed.  The  derivation  of  the  vari¬ 
ous  clock  rate  functions  is  also  provided.  To  calculate  the  clock  rate,  data 
has  to  be  measured  and  collected.  A  measurement  technique  has  to  be 
developed  to  obtain  the  data.  This  section  discusses  the  issues  associ¬ 
ated  with  developing  a  measurement  technique. 

The  definition  of  clock  rate  is  the  time  rate  of  change  of  the  clock. 
The  clock  rate  is  what  defines  the  lengths  of  time  for  a  unit  of  time  (e.g. 
second).  When  systems  have  different  clock  rates,  they  have  different 
units  of  time.  The  accumulation  of  the  difference  in  the  units  of  time  is 
called  the  clock  dr\ft.  Drift  is  part  of  the  total  difference  in  the  clock 
readings.  The  other  part  is  the  clock  offset.  The  clock  offset  is  the 
difference  in  the  clocks  at  the  initial  or  reference  time  (see  Figure  3A). 
The  offset  and  drift  both  appear  in  the  global  time  transformation  as  ao 
and  ai  in  Equation  (1)  respectively. 

The  rate  that  the  drift  is  changing  is  called  the  clock  drift  rate.  The 
clock  rate  and  the  drift  rate  are  different  (see  Figure  3A).  The  clock  rate  is 
calculated  using  the  clock  readings  with  respect  to  time.  The  clock  drift 
rate  is  calculated  as  the  change  in  drift  with  respect  to  time. 

The  Clock  Rate  Equation 

The  clock  rate  for  system  X  is  the  slope  of  the  time  line  for  the  system 
(the  thick  line  in  figure  3A).  The  clock  rate  for  system  X  is 
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Tx  = 


(2) 


_  t2.G  -  tl.G 
t2,X  -  tl.X 

where  t2,G  is  the  corresponding  global  time  for  the  local  time  t2  jc  for  clock 
X.  and  tl.G  is  the  corresponding  global  time  for  the  local  time  tix- 


Local  Clock  X 

Clock  rate  and  clock  Drift  rate 
Figure  3A 


The  clock  drift  rate,  dx .  for  system  X  is 
,  A  drift 

dx  =  - 

t2.x  -  tl.X 


A  drift  = 


Therefore, 


dx  = 


driftt^  -drifttj 

[t2,G-(t2.x  + clock  offset)]  -[tl.G -(ti.x  + clock  offset)] 
^2.G  “  f2.X  -  tl.G  +  tl.X 
(t2.G  -  tl.G)  -  (t2.X  -  tl.x) 

(t2/s-tin)~(t2,x-ti.x) 

t2JC-tl,X 


(3) 
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The  clock  rate  and  clock  drill  rate  are  different  but  are  related  in  the 
following  way. 

ti.x) 


(t2.X  ~tl.x) 
t2.X  “  tl.X 

1 


(4) 

For  the  global  clock,  the  clock  rate  is  ro  =  =  1 . 

t2.G  “  tl.G 

Substituting  this  into  equation  (4),  gives  you  the  following 

Tx  =  dx  +  re  . 

The  clock  rate  for  clock  X  is  the  clock  drift  rate  for  clock  X  plus  the  global 
clock  rate.  Notice  that  if  there  is  no  clock  drift  rate,  the  local  clock  rate  is 
the  same  as  the  global  clock  rate.  When  this  occurs,  the  clock  X  is 
accurate. 

The  clock  rate  is  the  rate  that  is  used  bv  the  global  time  trans¬ 
formation  as  the  coefficient  aj^.  This  is  because  clock  rate  simplifies  the 
calculations  in  the  transformation  and  it  is  easier  to  use  when  solving  the 
equations.  To  use  the  clock  drift  rate,  the  global  time  transformation 
would  have  to  be  modified  in  the  first  order  term  to 

G(ti^)  =  ao  +  (aj  +  l)*t^^  . 

In  this  report,  the  global  time  transformation  will  use  the  clock  rate  for  ai. 
To  calculate  the  clock  rate,  timestamp  data  is  gathered  from  each  of  the 
clocks.  This  can  be  accomplished  by  using  the  observer-source  model 
(see  figure  3B).  This  model  has  at  least  one  observer  that  monitors  each 
of  the  sources  during  an  experiment.  The  experiment  is  implemented  in 


dx  = 
dx  = 

Using  Equation 

dx  = 
or 

rx  = 


(t2.G- tl.G)~(t2.X 
t2.X  -  tl.X 

(t2.G  -tl.o) 
t2.X  -  tl,X 

(t2.G  ~  tl.o) 
t2.X  -  tl.X 

(2) 

dx+  1. 
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such  a  way  that  each  of  the  sources  communicates  their  present  clock 
time  to  the  observer.  The  observer  accepts  and  records  the  timestamp 
data  which  is  then  used  to  calculate  the  clock  rate. 

The  method  by  which  the  time  is  communicated  from  the  source  to 
the  observer(sj  is  dependent  upon  the  clocking  mechanism,  the  message 
type,  and  the  observer-source  arrangement.  The  clocking  mechanism  is 
the  mechanism  that  initiates  the  message.  The  message  type  is  the  kind 
of  message  that  is  sent.  The  observer-source  arrangement  defines  who 
sends  messages  to  who. 


Observer-Source  Model 
Figure  3B 


Clocking  Mgchaninmn 

The  four  basic  types  of  clocking  mechanisms  are  the  chimer,  the 
alarm,  the  pulser,  and  the  repeaterIMilh45I.  The  Chimer  sends /broad¬ 
casts  at  periodic  intervals  the  time  in  a  message  to  the  observer(s).  The 
message  contains  the  information  of  the  current  time.  This  is  like  a 
grandfather  clock  that  chimes  and  the  total  number  of  chimes  is  the  pre¬ 
sent  hour.  The  alarm  sends  at  a  specified  time  a  serial  to  the  ob- 
server(s).  A  signal  is  a  message  that  doesn't  contain  any  information 
about  the  time.  Because  the  time  that  the  signal  was  to  be  sent  was  pre¬ 
determined,  the  observer  only  needs  to  receive  a  signal  from  the  source  to 
know  the  departure  time  of  the  message.  Like  the  name  implies,  an 
alarm  is  like  an  alarm  clock  that  rings  at  a  specified  time.  The  ring 
doesn't  tell  what  time  it  is  but  the  listener  knows  because  the  signal 
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occurred  at  the  previously  specified  time.  The  pulser  sends/broadcasts 
at  periodic  intervals  a  sigrml  for  the  observer(s).  This  mechanism  is  pri¬ 
marily  used  to  synchronize  multiple  clocks.  The  repeater  sends/broad¬ 
casts,  only  upon  request,  the  time  or  a  signal  to  the  requester.  The  source 
acts  as  a  "pinging"  mechanism.  The  chimer,  alarm,  pulser  and  repeater 
are  the  basic  timing  mechanisms.  Selecting  the  appropriate  timing 
mechanism  will  depend  on  the  type  of  message  capability  in  the  source 
system. 


Source 


Chimer,  Alarm  and  Pulser  Model 


Repeater  Model 
Figure  3D 


static  and  Dynamic  Messages 

The  two  types  of  messages  are  the  static  and  the  dynamic  message 
types.  A  static  message  is  a  fixed  message  that  was  previously  defined 
and  built  for  the  observer(s).  Static  messages  are  used  primarily  as  sig¬ 
nals.  A  dynamic  message  is  a  message  that  is  generated  every  time  it  is 
requested.  The  dynamic  message  is  used  when  the  observer  needs  to  re¬ 
trieve  the  time  from  the  source's  clock.  Some  sources  don't  have  the 
ability  to  generate  dynamic  messages.  The  type  of  message  will  depend 
on  the  experiment,  the  source  and  the  clocking  mechanism. 

Obgervcr-Source  Arrangcmcntg 

The  third  section  of  the  methodology  is  the  observer-source  arrange¬ 
ment.  This  subsection  identifies  to  whom  the  message  is  to  being  sent. 
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The  three  fundamental  arrangements  are  the  singular,  multicast  and 
broadcast  arrangements.  In  the  singular  arrangement,  the  source  sends 
the  time  or  a  signal  to  only  one  observer.  In  the  multicast  arrangement, 
the  source  sends  the  time  or  signal  to  a  select  set  of  observers.  In  the 
broadcast  arrangement,  the  source  indiscriminately  sends  the  time  or 
signal  to  all  observers.  Selecting  the  appropriate  observer-source  ar¬ 
rangement,  the  clocking  mechanism  and  the  message  type  help  define  the 
measurement  technique  to  be  used  for  the  experiment. 

Experiment  Configurations 

Using  the  observer-source  model  and  the  various  communication 
methods,  there  are  many  types  of  experiments  that  can  be  developed  to 
gather  the  data.  Only  two  of  these  experiments  will  be  discussed  in  this 
report.  The  first  experiment  involves  a  source  that  has  the  ability  to  gen¬ 
erate  d3mamic  messageslMills  92).  If  dynamic  messages  and  the  chimer 
mechanism  are  used,  the  source  would  extract  at  periodic  intervals  (e.g. 
once  an  hour  or  day)  the  time  from  the  system  clock  put  it  into  a  mes¬ 
sage,  then  send  the  message  to  the  observer(s).  The  observer(s)  would 
then  extract  the  time  from  the  message  and  records  both  the  time  datum 
within  the  message  and  the  arrival  time  of  the  message.  If  dynamic  mes¬ 
sages  and  the  repeater  mechanism  are  used,  the  observer  would  send  a 
request  message  to  the  source.  TTie  source  would  receive  the  message 
from  the  observer,  get  the  time  from  the  system  clock,  put  it  into  a  mes¬ 
sage  and  then  send  it  to  the  observer.  Again,  the  observer  would  receive 
the  message,  extract  the  time  from  the  message  and  record  both  the  de¬ 
parture  time  and  the  arrival  time  of  the  message.  The  main  advantage  of 
the  dynamic  message  technique  is  that  the  message  contains  departure 
time  information  while  a  static  message  doesn't.  The  main  disadvantage 
of  the  dynamic  message  technique  is  that  it  takes  time  to  extract  the  time 
from  the  clock  and  put  into  the  message.  This  additional  time  distorts 
the  timestamp  data.  If  the  time  between  messages  is  very  long,  the 
distortion  has  less  of  an  impact  on  the  calculation  of  the  clock  rate. 

In  the  second  experiment,  the  source  only  has  the  ability  to  send 
static  messages(i.e.  signals)  and  the  clocking  mechanism  is  the  alarm. 
The  alarm  mechanism  is  set  to  a  series  of  specified  times.  When  the 
specified  times  arrive,  the  source  sends  a  signal  (i.e.  static  message)  to  the 
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observer(s).  The  observer(s),  upon  receiving  the  message,  records  the  ar¬ 
rival  time.  The  arrival  time  is  not  the  departure  time  for  the  message. 
The  arrival  time  is  the  departure  time  plus  all  the  propagation  delays 
between  the  systems.  In  the  dynamic  message  case,  the  observer  gets 
both  the  departure  time  and  the  arrival  time.  The  main  advantage  of  this 
experiment  is  that  it  is  easy  to  implement.  The  main  disadvantage  is  that 
it  is  hard  to  account  for  all  those  fluctuating  delays  when  calculating  the 
clock  rate.  There  are  more  experiments  that  can  be  generated,  but  these 
two  show  how  dynamic  and  static  messages  can  be  used  to  provide 
timestamp  data  to  the  observer  . 

Periodic  Intervals 

An  issue  that  needs  to  be  addressed  is  how  to  generate  periodic  inter¬ 
vals  for  the  chimer  and  pulser  clocking  mechanisms.  There  are  two  fun¬ 
damental  techniques  to  generating  periodic  intervals  with  software.  One 
technique  is  to  have  the  software  enter  a  wait  state  until  the  timer  counts 
down  to  zero,  then  leaves  the  wait  state,  sends  the  message,  resets  the 
timer  to  the  specified  interval  time  and  returns  again  to  the  wait  state. 
The  process  is  repeated  over  and  over  again  during  the  experiment.  This 
technique  is  the  wait-state  technique.  The  problem  with  the  wait-state 
technique  is  that  all  that  time  spent  outside  the  wait  state  is  added  to  the 
cycle  time.  The  actual  time  for  one  cycle  is  the  interval  time  plus  the  ad¬ 
ditional  time  spent  outside  the  wait  state  while  the  software  is  running  in 
between  the  wait  states.  The  additional  time  outside  the  wait  state  accu¬ 
mulates  to  noticeable  levels  within  seconds.  To  compensate  for  the  ac¬ 
cumulated  effects  of  the  software  processing  time  to  the  interval,  the  in¬ 
terval  time  for  the  wait  state  is  reduced.  This  adjustment  to  the  wait- 
state  time  isn't  exact  because  of  the  variations  in  the  processing  time  for 
the  message  and  resetting  the  timer. 

The  other  technique  is  to  have  the  source  software  use  the  system 
clock  and  at  the  specified  clock  time(s)  the  software  sends  a  message  to 
the  observer.  When  the  message  arrives,  the  observer  records  the  arrival 
time.  The  time  between  messages  is  based  on  the  clock  time  and  not  an 
interval  of  time.  This  technique  is  the  clock-lxised  technique.  The  prob¬ 
lem  with  the  clock-based  technique  is  that  the  software  keeps  checking 
the  clock  to  see  if  the  time  has  expired.  When  the  processor  spends  time 
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to  check  the  clock,  it  takes  the  processor  away  from  the  other  processing 
activities.  This  slows  down  the  primary  processing  activities.  To  correct 
this  problem,  separate  mechanism  is  needed  to  monitor  the  clock  so  that 
the  processor  is  not  interrupted.  Because  there  isn't  any  accumulative 
effects  with  the  clock-based  technique,  the  clock-based  technique  is  the 
better  technique  of  the  two  techniques  for  generating  periodic  intervals. 

Long  Interval  and  Short  Interval  Techniques 

After  gathering  the  timestamp  data,  the  next  step  is  to  calculate  the 
clock  rate.  The  two  fundamental  techniques  for  calculating  the  clock  rate 
are  referred  to  as  the  long  interval  technique  and  the  short  interval  tech¬ 
nique  (see  figure  3C).  With  the  long  interval  technique,  the  data  (i.e., 
ti.G.  t2.G.  tijc  and  t2jd  are  collected  over  a  long  period  of  time  and  then 
the  initial  measurements  and  the  final  measurements  are  substituted  into 
Equation  (2)  to  calculate  the  clock  rate.  With  the  short  interval 
technique,  the  data  is  collected  over  a  shorter  period  of  time.  From  the 
sets  of  interval  measurements,  a  set  of  interval  clock  rates  are  calculated. 
From  the  set  of  interval  clock  rates,  the  ejffective  clock  rate  is  calculated 
for  the  system.  Of  the  two  techniques,  the  simplest  and  most  precise 
technique  is  the  long  interval  technique  because  the  variations  in  data 
have  less  of  an  impact  on  the  calculation.  Because  the  size  of  the 
variations,  (fractions  of  a  millisecond),  is  relatively  small  compared  to  the 
measurement  interval,  (weeks  or  months),  the  results  of  calculations  are 
more  accurate.  One  problem  with  using  the  long  interval  technique  is 
that  it  is  very  hard  to  get  a  system  to  stay  on  long  enough  for  a  complete 
set  of  measurements,  to  be  taken.  Usually  there  is  a  problem  that 
interrupts  the  system.  For  example,  the  system  is  rebooted  because  of  a 
software  problem  or  the  system  power  is  turned  off  either  manually  or  by 
a  power  outage.  The  short  interval  technique  is  used  when  the  window  of 
time  in  which  you  have  to  measure  the  clock  rate  is  relatively  small  (e.g.  a 
couple  of  days).  The  problem  with  this  technique  is  that  all  the  variations 
in  the  delays  that  occur  during  the  measurements  have  to  be  accounted 
for  in  the  calculation  of  the  clock  rate.  The  types  of  delays  are  numerous 
and  hard  to  measure.  Some  of  the  delays  are  in  the  message  firing  times, 
the  communication  network,  the  system  loading,  encoding/decoding  the 
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network  protocols  and  the  measurement  process.  To  make  the  problem 
worse,  the  delays  vary  in  time  and  therefore  are  hard  to  determine. 


Long  Interval  Technique  Short  Interval  Technique 

Long  interval  and  Short  interval  Calculations 
Figure  3C 

In  the  long  interval  technique,  the  steps  are  simply  to  take  the  initial 
measurements,  take  the  final  measurements  and  put  them  into  equation 
(2)  (see  Figure  3C).  The  short  interval  technique  is  not  quite  as  simple 
(see  Figure  3C).  Many  more  measurements  required  because  the  calcula¬ 
tions  are  sensitive  to  measurement  errors  and  variations  in  the  delays. 
To  make  the  calculations  more  accurate,  the  equation  (s)  for  the  clock 
rate  must  be  expanded  to  take  into  account  the  variations  in  the  mea¬ 
surement  process  (e.g.  network  effects  and  system  loading). 

A  Four  Clock  System 

The  clock  rate  calculations  are  dependent  on  the  number  of  delays 
that  are  encountered  during  the  message  passing  process.  The  number 
of  delays  depend  mainly  on  the  amount  network  hardware  there  is 
between  the  message  source  and  the  observer.  To  derive  the  equations 
for  the  clock  rate,  a  four  clock  system  model  is  used.  Figure  3D  shows 
the  interconnectivity  of  the  system  model.  Clocks  Y  and  Z  are  on  the 
opposite  side  of  the  network  than  the  global  clock  G.  Clock  X  is  on  the 
same  side  of  the  network  as  the  global  clock  G.  Clock  Y  and  X  are  a  static 
message  devices  and  clock  Z  is  a  d5mamic  message  device. 
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Figure  3E  is  used  to  help  track  the  movement  of  the  two  messages  as 
they  go  from  source  to  observer.  The  flow  of  two  messages  from  system  Z 
to  system  G  is  depicted  by  the  two  groups  of  shaded  areas.  In  both  the 
upper  and  lower  shaded  areas,  area  a  signifies  the  time  that  the  message 
takes  to  go  from  the  network  to  the  application,  area  n  signifies  the  time 
that  the  message  is  in  the  network  and  area  b  signifies  the  time  that  it 
takes  to  go  from  the  application  to  the  network.  The  elapsed  global  time 
of  each  event  is  read  off  of  the  vertical  axis.  The  elapsed  local  time  for  an 
event  is  read  off  of  the  horizontal  axis. 

Let's  identify  the  recorded  data  for  the  first  message,  which  is  tlie 
lower  shaded  area.  Message  one  is  Sent  from  system  Z  at  time  Si  ,z.  Ac¬ 
cording  to  system  Y.  the  message  Enters  the  network  at  time  Ei.y.  Ac¬ 
cording  to  the  clock  in  system  X,  the  message  Leaves  the  network  at  time 
Li^.  The  message  is  Received  by  system  G  at  time  Ri.c-  Four  separate 
time  lines  are  drawn  for  the  four  separate  clocks.  The  time  lines  for 
clocks  X,  Y  and  Z  represent  measured  drift  data  and  are  arbitrarily  se¬ 
lected  for  the  derivation  of  the  clock  rate.  Figure  3E  will  be  used  in  the 
derivation  of  the  clock  rate  for  the  three  systems.  The  numeric  entries  on 
the  plots  are  not  used  in  the  derivation  but  can  aid  in  following  the 
derivation. 

The  notation  that  is  used  in  the  plot  and  in  the  derivations  of  the 
clock  rates  is  as  follows; 

T  is  the  timestamp  datum  or  reading  from  the  system  clock, 
t  is  the  elapsed  time. 

•  is  a  recorded  timestamp  datum. 
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Ei^  is  the  length  of  time  it  takes  for  the  i^l^  message  to  go  from  the 
network  to  the  application  with  respect  to  clock  A. 

is  the  length  of  time  it  takes  for  the  i^h  message  to  go  from  the 
oDDlication  to  the  network  with  respect  to  clock  A. 

Ei^  is  the  elapsed  time  with  respect  to  clock  A  when  the  i^ 
message  Enters  the  network. 

Li^  is  the  elapsed  time  with  respect  to  clock  A  when  the  i^^^ 
message  Leaoes  the  network. 

nijv  is  the  length  of  time  (network  propagation  delau)  it  takes  for  the  i^l^ 

message  with  respect  to  clock  A. 

Oa.g  is  the  Offset  of  clock  A  with  respect  to  clock  G. 

rA.B  is  the  rate  that  clock  A  is  drifting  from  clock  B. 

Si^  is  the  elapsed  time  with  respect  to  clock  A  of  when  the  i^^ 
message  was  Sent. 

Ri^  is  the  elapsed  time  with  respect  to  clock  A  of  when  the  ii^ 
message  was  Received. 


In  Figure  3E,  there  are  two  types  of  variables,  those  with  recorded 
data  and  those  without  recorded  data  e.g.,  E2,y  and  Ei,g  respectively. 
The  •  for  E2,y  signifies  that  it  is  a  recorded  value.  Ei,g  is  not  a  recorded 
datum  because  it  doesn't  have  a  •. 

Derivation  of  the  Clock  Rate  Equations 

There  are  only  three  clock  rates  that  have  to  be  calculated  because 
one  of  the  clocks  is  the  reference  or  global  clock.  The  three  clock  rates 
are  rx,  ry.  and  rz-  By  definition, 

.  (5) 

L2.X  -  Li.x 
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Since  Liq  *  Ri.c  -  ai.o  and  L^.g  *  Ra.c  -  a2.G  where  ai.G  is  the  length  of 
time  it  takes  the  i^  message  to  propagate  from  the  network  to  the  appli¬ 
cation  as  measured  by  clock  G 

L2.G  -  Li.g  =  (R2.g  -  a2.G)  -  (Ri.g  -  am) 

=  (R2.g  -  Ri.g)  -  (a2.G  -  ai.G) 

Substituting  this  into  Equation  (5) 

(R2.g  ~  Ri.g)  ~  (a2.G  ~  ai.n) 

L2.X  -  Li.x 

(R2.g  -  Ri.g)  _  (a2.G  ~  am) 

L2.X  -  Lijc  L2.X  ~  Li.x 

Note:  From  the  data  available,  it  is  not  possible  to  directly  calculate  a2,G' 

ai.G. 


a2.G  “  ai.G  *  L2.G  “  Li.g 

=  (R2.g  -  L2.g)  ~  (Ri,g  “  Li.g) 

=  (R2.g  -  Ri.g)  “  (L2.G  -  Li.g) 

Since  Li.g  =  rx*Li.x  +  Ox.g  and  L2.G  =  rx*L2.x  +  Ox.g  then  to  calculate 
L2.G  -  Li.g  the  value  for  rxmust  be  approximated  because  it  is  not  known. 
The  approximation  for  rx  is  used  to  calculate  L2.G  -Li.g  which  is  then  used 
to  calculate  a2.G  -  ai.G-  After  substituting  a2.G  -  ai.G  Into  Equation  (6),  the 
resulting  calculation  is  equal  to  the  value  that  was  selected  for  the  ap¬ 
proximation.  If  R2.G  -  Ri.g  »  a2.G  *  ai.G  then 


R2.G  -  R1.0 
L2.X  -  Li.x 


(7) 


This  is  a  good  assumption  because  the  size  of  ai.G  is  relatively  smaller 
than  the  network  delay  and  therefore  ^+i.g  -  ai.G  is  very  small,  since  ai+i.G 
and  ai.G  both  are  >  0. 
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The  next  clock  rate  to  drive  is  ry.  The  calculation  of  the  clock  rate 
for  S3rstem  Y  starts  with  the  definition  of  clock  rate, 


_  E2,g  ~  Ei.g 

ry  =  - ! - 

E2.Y  -  Ei.y 


Since  Ea.G  *  R2,g  ■  £t2,G  ~  n2.G  aiid  Ei,g  =  Ri.g  ■  ai.c  ■  ni,G  then 

E2.G  -  Ei.g  =  (R2.g  ~  a2.G  -  n2.G)  ~  (Ri.g  ~  ai.G  “  ni.G) 

=  (R2.G  -  Ri.g)  ~  (a2.G  ~  ai.G)  ~  (n2.G  -  ni.G)  •  (9) 

Substituting  this  into  Equation  (8) 

_  (R2.g  ~  Ri.g)  ~  (a2.G  ~  ai.o)  ~  (n2.G  -  ni.o) 

E2.Y  -  Ei.y 


(R2.g  -  Ri.g) 
E2.Y  “  Ei.y 


(a2.G  ~  ai.G)  _  (n2.G  -  hi.g) 


E2,Y  ”  Ei,y 


E2.Y  -  Ei.y 


.  (10) 


Since  a2.G‘ai,G  is  relatively  small,  the  Equation  (9)  can  be  approximated 
by 


(R2.g  ~  Ri.g) 
E2.Y  -  Ei.y 


(n2.G  ~  ni.G) 

E2.Y  -  Ei.y 


The  only  unknown  variable  in  this  equation  is  n2,G-ni.G-  Since  n2.G  =  L2.G 
-  E2.G  and  n2.G  =  L2.G  -  E2.G  then 

n2.G  “  ni.G  =  (L2.g  -  E2.g)  -  (Li.g  -  Ei.g) 

=  (L2.g  -  Li.g)  -  (E2.G  -  Ei.g)  (12) 


Using  the  definitions  for  rx  and  ry,  L2,g  -  Li.g  =  rx*(L2.x  -Li.x)  and 
E2.G  -  Ei.g  =  ry*(E2.y  -  Ei.y).  Substituting  these  into  Equation  (12) 

n2.G-ni.G  =  rx*(L2.x  -  Li.x)  -  ry  •(E2.Y  -  Ei.y)  • 
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The  term  (n2.G  -  ni.c)  is  the  change  in  the  network  propagation  times  be¬ 
tween  messages  2  and  1.  The  purpose  of  this  term  in  Equation  (10)  is  to 
eliminate  the  effects  of  the  variation  in  the  network  propagation.  Since 
(n2,G  -  Hi.g)  is  dependent  on  Tx  and  ry  which  are  unknown,  approximate 
values  for  rx  and  ry  are  used  (i.e.  rx  and  ry  respectively),  see  Figure  3F. 
The  better  the  approximation  for  rx  and  ry  the  better  the  resulting  cal¬ 
culation  for  rx  and  ry.  Which  value  to  use  for  the  approximation  is  up  to 
the  analyst.  A  couple  of  candidate  approximations  for  the  clock  rates  are 
the  last  calculated  clock  rate  or  the  clock  rate  derived  from  the  initial  and 
final  measured  values.  Since  the  error  in  the  approximation  to  the  Anet- 
work  propagation  is  smaller  then  the  A  network  propagation,  the 
adjustment  with  the  approximated  clock  rate  reduces  the  impact  of  the 
variations  in  network  effects.  This  will  provide  a  much  better  value  for 
the  clock  rate,  see  Figure  3F.  Since  (E^.y  -  Ei,y)  is  a  factor  of  the  term 
ry*(E2,Y  -  Ei,y),  the  linear  matrix  equation  solution  to  the  problem  is  not 
feasible.  The  iE2,Y  -  Eiy)  factor 
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would  eliminate  the  ry  variable  (see  Equation  (15)).  The  approximation  for 
(n2,G  -  hi.g)  is 

n2.G-ni,G  “  r'x  *  (L2,x  -  Li.x)  -  r'y  •(E2.y  “  Ei.y)  .  (14) 
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Substituting  this  into  Equation  (12) 

^  _  (R2.0  -  R..c)  _  (r"x  *  (L2.x  -  Li.x)  ~  t'y  *  (E2.Y  -  Ei.y)) 

E«  -  Ei.r  E2.Y  -  Ei.y 


Ea.T  Ei.t 


(Rs.g-Ri.g)  -  r  jr  •  (L2.x  -  Li.x)  -  r  y*(E2.y  “  Ei.t) 

E2.Y  -  Ei,y 

The  next  clock  rate  to  drive  is  rz-  The  calculation  of  the  clock  rate 
for  system  Z  starts  with  the  definition  of  clock  rate. 


=  ^2.g  ~  Sl.G 

S2.Z  -  Si.z 


Since  S2.G  =  Ez.c  -  b2.G  and  Si,g  =  Ei,g  -  bi.G  then 

Sajo  ~  Siij  —  (E^g  ~  bajo)  ~  (Ei,g  ~  bus) 

=  (Eax!  ~  Es.o)  ~  (baxj  ~  bie)  • 

Substituting  this  into  Equation  (16) 

(E2.g  -  Ei.g)  -  (b2,G  -  bi.G) 

rz  =  ^ - — 

S2,z  “  Sl,Z 

(E2.G  -  Ei,g)  _  (b2.G  ~  bl.u) 

S2.Z  ~  Si.z  S2Z  ~  Si^ 


Substituting  Equation  (9)  for  Ez.g  -  Ei.g 

_  (Rz.g  -  Ri.g)  -  (a2.G  -  ai.G)  -  (nz.G  ~  ni.u)  _  (bz.G  -  bi.G) 

S2,z  -  Si,z  S2,Z  “  Si.z 


(Rz.g  -  Ri.g)  _  (a2,G  -  ai.G)  _  (nz.G  ~  gi.g)  _  (b2.G  -  bi,G) 
S2.Z  “  Si.z  S2.Z  ~  Si.z  S2,z  ~  Si.z  S2.2  ~  Si.z 
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(R2.g  ~  Ri.g) 
S2.Z  “  Si,z 


(nz.G  ~  iii.g) 
S2.Z  ~  Si.z 


(a2.G  ~  ai.c) 
S2JZ  -  Si^ 


(b2.G  ~  bi.c) 

S2,Z  “  Si^ 


(18) 


Like  a2.G“  ai.c.  b2.G  -  bi.o  is  relatively  small.  The  only  difference  between 
a2,G-  ai.G  and  b2,G  -  bi.o  is  that  a2.G'  ai,G  is  the  difference  in  the  length  of 
time  it  takes  for  a  message  to  go  from  the  network  to  the  application  and 
b2.G  -  bi,G  is  the  difference  in  the  length  of  time  it  takes  for  a  message  to 
go  from  the  application  to  the  network.  Since  a2,G-ai.G  and  b2,G  -  b-.G  are 
relatively  small,  the  Equation  (18)  can  be  approximated  by 


(Ra.o  ~  Ri.g) 

Saz  ~  Siz 


(na.o  “  ni.c) 
Saz  ~  S’z 


(19) 


The  only  unknown  in  this  equation  is  n2.G-n1.G-  Using  Equation  (14)  into 
Equation  (19)  to  get 


Tz 


(Ra,o-Rix: 

s«-s.; 


(r'x  •  (Lajc  ~  Lijc)  ~  T'i  *  (Ea.Y  ~  Ei.y)) 
Saz  ~  Siz 


rz 


(Ra.o  —  R1.0)  ~  •  (l/a.x  —  Li.z)  ~  t'r  *  (E»a.T  ~  Ei.t) 

S3.Z  ~  Siz 


(20) 


The  Clock  Rate  Eauationg 

The  clock  rate  equations  derived  in  this  section  are  the  equations 
that  are  needed  to  calculate  the  ai  coefficients  of  the  global  time  trans¬ 
formation.  These  equations  are 


(R2,g“Ri,g)  r  ^  ^ 

rx  “  - - -  for  R2.g-Rvg  »  a2,G-ai,G 

L2,x  -  Li,x 


(21) 


ry  * 


(R2.g-Ri.g)  -  ~  -  r'y  *  (E2.y  -  Ei,y) 

E2.Y  -  Ei,y 


(22) 


(R2.G  -  Ri.g)  -  r  •  (L2.X  -  Li.x)  -  r  y  *  (E2.y  -  Ei.y) 

rz  *  - - — - - - -  -  (23) 

S2.Z  ~  Si.z 
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Equation  (21)  is  used  when  the  data  comes  from  a  monitoring  system 
on  the  same  side  of  the  network  as  the  global  clock.  Equation  (22)  is  use 
when  the  data  comes  from  a  monitoring  system  on  the  other  side  of  the 
network  from  the  global  clock.  Equation  (23)  is  used  by  a  computer  sys¬ 
tem  on  the  other  side  of  the  network.  In  the  next  section,  the  other  coeffi¬ 
cient  for  the  global  time  transformation,  ao  is  discussed. 
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Pcflnitfwt 

This  section  describes  a  technique  for  measuring  and  calculating  the 
clock  offset.  Since  the  clock  offset  is  sensitive  to  delays,  a  detail 
breakdown  of  the  dela3rs  in  the  message  path  is  given  for  both  the  static 
and  dynamic  message  techniques.  The  later  part  of  the  section  discusses 
the  tradeoffs  between  the  static  and  d3mamic  message  technique,  which 
will  help  the  reader  select  the  proper  technique  for  measuring  the  clock 
offset. 

In  the  Global  Time  Transformation,  there  are  two  coefficients.  The 
coefficient  ai  is  the  clock  rate  which  was  discussed  in  Section  3.  The 
other  coefficient  ao  is  the  clock  offset  and  is  the  topic  of  this  section.  The 
definition  of  clock  tfffset  is  the  difference  in  time  from  the  local  clock 
reading  to  the  global  clock  reading  at  the  initial  or  reference  time  (see  Fig¬ 
ure  4A). 

Once  the  clock  offset  is  measured,  it  can  be  used  with  the  clock  rate 
to  form  the  global  time  transformation.  The  accuracy  of  the 
transformation  depends  on  the  accuracy  of  the  clock  offset  measurement. 
Measuring  the  clock  offset  to  be  correct  to  within  a  specified  error  (e.g.  1 
millisecond)  is  more  difficult  to  do  than  the  clock  rate.  The  difficulty  is 
that  the  clock  offset  measurements  are  taken  within  a  few  seconds,  unlike 
the  clock  rate  which  is  hours  or  days. 

The  clock  rate  measurements  are  usually  taken  at  the  beginning  of 
the  experiment.  Since  the  clocks  are  aligned  at  the  beginning  of  the 
experiment,  the  offset  should  be  mes^ured  right  after  the  alignment.  This 
is  primarily  due  to  the  fact  that  the  magnitude  of  the  drift  effects  are 
smallest  at  that  the  beginning  of  the  experiment.  The  clock  rate  defines  a 
set  of  global  time  transformations  for  that  system  and  the  clock  offset 
determines  which  one  of  the  set  is  the  particular  transformation  for  this 
experiment  (see  Figure  4A).  Since  there  is  a  different  clock  offset  for  every 
experiment,  every  experiment  has  a  different  global  time  transformation. 
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Figure  4A 

The  clock  rate  only  needs  to  be  measured  once  if  the  operational  en¬ 
vironment  for  the  clock  is  relatively  constant  or  the  experiment  has  a 
short  time  duration.  A  new  clock  offset  must  be  measured  for  each  exper¬ 
iment  or  when  the  clock  is  changed.  A  change  to  the  clock  can  occur  in¬ 
tentionally  (e.g.  a  time  daemon  or  a  manual  system  shutdown)  or  unin¬ 
tentionally  (e.g.  power  outage  due  to  lightning  or  an  automatic  system 
shutdown  if  the  temperature  gets  too  hot).  When  the  data  for  the  experi¬ 
ment  is  collected  both  before  and  after  a  system  interruption,  the  analyst 
must  use  two  global  time  transformations  for  the  transformation  step. 
One  time  transformation  for  the  data  collected  before  the  interruption  and 
another  time  transformation  for  the  data  collected  after  the  interruption. 

Clock  Offset  Tcchnioue 

The  clock  offset  is  measured  using  either  a  static  or  a  dynamic 
message  technique.  The  static  and  dynamic  techniques  used  to  calculate 
the  clock  offset  are  different  than  those  used  for  the  clock  rate.  The  static 
message  technique  used  for  the  clock  offset  uses  both  the  interval  time 
and  the  message  propagation  measurements,  unlike  the  clock  rate  which 
uses  only  the  interval  time  measurements  (see  Figure  4B).  The  reason  is 
that  the  clock  rate  is  a  function  of  the  change  in  time  while  the  clock  off¬ 
set  is  a  length  of  time.  The  message  propagation  measurement  is  needed 
to  calculate  the  clock  offset. 
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Message  Propagation 

The  reason  for  measuring  the  message  propagation  is  that  the  ob¬ 
server  can  calculate  the  departure  time  of  the  interval  message  from  the 
arrival  time  of  the  interval  message  if  the  observer  knows  how  long  it  took 
the  message  to  get  to  the  observer.  The  calculation  is 

tArrtvai.G  “  tDeparture.G  Message  Propagation^ 

tDeparture.G  “  tArrtvai.G  ”  Message  Propagation^  (21) 

Where  toeparture.G  soid  tArrtvai.G  the  message  departure  and  arrival 
times  respectively,  in  global  time  units,  and  the  Message  Propagation 
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Figure  4B 


is  the  length  of  time,  in  global  clock  units,  that  the  message  takes  to  go 
from  the  source  to  the  observer.  Note:  The  calculated  departure  time  is  in 
global  clock  time  units  which  is  different  than  the  departure  time  in  clock 
X  time  units.  This  difference  is  noticeable  when  the  d)aiamic  message 
technique  is  used  because  the  message  contains  the  departure  time  in 
clock  X  time  units.  The  departure  times  extracted  from  the  dynamic  mes¬ 
sage  will  be  different  from  the  calculated  departure  time  because  the 
clock  time  units  are  different.  The  next  step  is  to  measure  the  message 
propagation  delay. 
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To  measure  the  message  propagation,  a  test  message  is  sent  from  the 
observer  to  the  source  and  the  source  returns  the  message  to  the  observer 
(i.e.  the  observer  "pings"  the  source)  and  the  round  trip  time  is  measured. 
The  simplest  method  for  calculating  the  message  propagation  is  to  divide 
the  round  trip  time  by  two.  Since  the  propagation  in  one  direction  is  not 
equal  to  the  propagation  in  the  other  direction,  the  approximation  for  the 
message  propagation  is  veiy  rough.  The  approximation  is  rough  because 
the  propagation  delays  vary. 

To  get  a  more  precise  value  for  the  message  propagation,  a  set  of 
measurements  are  taken  and  used  to  calculate  the  message  propagation. 
Between  the  interval  time  measurements  (see  Figure  4B).  another  set  of 
measurements  are  collected  which  are  the  trip  times  for  messages  going 
from  the  observer  to  the  source  and  return.  The  number  of  round  trip 
time  measurements  varies  quite  a  bit  depending  on  the  distance  between 
the  source  and  the  observer,  number  of  time  intervals  and  the  width  of 
the  time  intervals.  Since  most  systems  are  stationary,  the  distance  be¬ 
tween  the  source  and  observer  is  fixed.  Therefore,  only  the  width  of  the 
intervals  and  the  number  of  intervals  are  imder  your  control.  The  length 
of  time  between  the  interval  times  must  be  wide  enough  that  an  adequate 
number  of  measurements  can  be  taken  (see  Figure  4B).  The  number  of 
measurements  must  be  large  enough  to  show  the  true  variability  of  the 
message  propagation.  Larger  time  intervals  allow  for  more  message  prop¬ 
agation  measurements.  In  our  experiments,  the  observer  and  server  were 
relatively  close,  so  our  set  of  measurements  was  defined  to  be  a  collection 
of  ten  intervals  with  a  width  of  one  second  and  averaged  approximately  10 
round  trip  times  (i.e.  pings)  per  interval.  The  total  number  of  propagation 
measurements  were  100  for  the  experiment. 

Once  the  measurements  have  been  taken,  the  analyst  has  multiple 
methods  for  approximating  the  message  propagation.  One  method  is  to 
observe  the  message  propagation  times  that  where  taken  before  and  after 
the  interval  time  measurement.  Select  those  intervals  where  the  message 
propagation  measurements  before  and  after  the  interval  time  measure¬ 
ment  are  relatively  constant.  If  the  times  are  constant  or  nearly  constant, 
then  you  can  assume  that  dividing  the  propagation  by  two  is  a  good  ap¬ 
proximation  to  the  message  propagation.  The  hidden  assumption  is  that 
the  message  propagation  in  one  direction  is  the  same  as  the  other  direc- 
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tion.  This  is  not  the  case  even  though  the  round  trip  times  around  a  time 
interval  measurement  are  constant.^  In  general,  message  propagation 
time  is  direction  dependent  If  the  analyst  assumes  that  the  propagation 
is  the  same  in  both  directions,  the  approximation  will  be  in  error.  The 
amount  of  the  error  in  the  approximation  is  dependent  on  the  relative  size 
of  the  difference  in  the  propagation  times  and  the  length  of  the  message 
propagation.  The  multiple  message  measurements  give  the  analyst  a 
means  for  developing  confidence  in  the  propagation  approximation. 

To  get  an  even  better  approximation  for  the  message  propagation, 
statistical  methods  can  be  used.  Since  the  in-house  effort  had  cost  and 
manpower  limits,  work  in  this  area  was  not  performed.  For  that  reason, 
the  statistical  analysis  method  will  not  be  discussed  in  this  report. 

Static  Measage  Technique 

Once  the  message  propagation  has  been  determined,  the  next  step  is 
to  calculate  the  clock  offset.  By  definition  the  clock  offset  is 


Clock  Offset  =  ti.G  -  ti.x 


where  t(,G  is  the  time  event  i  occurred  according  to  clock  G  and  tpi  is  the 
time  event  i  occurred  according  to  clock  X-  If  the  event  i  is  the  departure 
of  an  interval  time  message  from  the  source,  then  t(.G  is  tDcparture.G  U-e.  the 
departure  time  of  the  interval  time  message  according  to  clock  G),  and  t(^ 
is  tDeparture,x-(i-e.  the  departure  time  of  the  interval  time  message  accord¬ 
ing  to  clock  JQ.  The  clock  offset  in  terms  of  the  departure  times  of  the  in¬ 
terval  time  message  is 

Clock  Offset  =  tDcparture.G  ~  tDcparture.X  • 


^  If  the  message  propagation  is  constant  then  the  round  trip  times  will  be  constant 
The  inverse  is  not  true. 
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Substituting  Equation  (21)  into  this  equation 

Clock  Offset  =  (tArTivai.G‘  Message  Propagationc)  -  toeparturejc  (22) 

tArrtvaiG  is  the  Observed  arrival  time  for  the  message.  tDepartunijc  is  the  pre¬ 
determined  departure  time  for  the  Interval  time  message.  For  example,  if 
the  message  departs  at  1:00  PM  and  arrivals  at  1:06  PM  and  the  message 
propagation  is  a  minute,  then  the  clock  offset  is  5  minutes. 

To  make  the  calculation  of  the  message  propagation  even  more  accu¬ 
rate.  a  refinement  can  be  made  to  the  departure  time.  The  refinement 
compensates  for  the  fact  that  the  source  does  not  give  an  instantaneous 
response  to  the  ping  message.  There  is  a  delay  within  the  source.  The 
delay  is  broken  up  into  three  parts.  The  first  part  is  the  length  of  time 
that  it  takes  to  sense  the  ping  message.  The  second  part  is  the  time  that 
it  takes  to  determine  how  to  respond  to  the  ping  message.  The  third  part 
is  executing  the  response  to  the  ping  message.  The  total  time  for  all  three 
parts  can  be  significant  enough  to  impact  the  precision  of  the  clock  offset 
calculation.  Figure  4C  shows  the  response  time  of  a  LAN  Protocol  Ana- 
l3rzer  that  was  a  source  during  the  clock  offset  measurements  using  static 
message  technique. 

The  static  message  technique  is  a  good  technique  for  calculating  the 
clock  offset  because  additional  measurements  can  be  taken  to  refine  the 
calculation  of  the  current  message  propagation  time.  Furthermore,  the 
static  message  technique  provides  many  opportunities  to  improve  the 
precision  of  the  clock  offset  calculation  (e.g.  statistical  methods). 

Dynamic  Message  Technique 

The  dynamic  message  technique  is  much  simpler  but  is  not  as  pre¬ 
cise  as  the  static  message  technique.  In  the  dynamic  message  technique 
(see  Figure  4D).  the  observer  gets  the  present  time  from  the  observer's 
clock  and  puts  it  into  a  message(l).  The  message  is  then  sent  out  over  the 
network  to  the  source.  The  source  receives  the  message,  gets  the  arrival 
time  of  the  message  for  the  clock  and  puts  it  into  the  incoming  message 
with  the  departure  time(2).  Since  it  takes  time  to  process  the  message  at 
the  source,  the  source  gets  a  new  departure  time  and  puts  it  into  the 
message  with  the  other  departure  time  and  arrival  time(3).  The  message 
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is  then  sent  out  over  the  network  to  the  observer.  The  observer  receives 
the  return  message,  gets  from  the  clock  the  arrival  time  of  the  returning 
message  and  puts  it  into  the  message  and  stores  it  away(4). 

The  stored  departure  and  arrival  times  are  used  to  calculate  the  clock 
offset  [Mills  92]  as  follows: 

Clock  Offset  =  (^Amval.2  ~  tDeparture.2)  +  (tAn1val.l  '  tpeparture.l)  . 

2 

This  result  is  a  rough  approximation  to  the  clock  offset  because  it  as¬ 
sumes  that  the  propagation  in  both  directions  are  the  same.  The  calcu¬ 
lated  value  has  just  about  the  same  accuracy  as  the  static  message  tech¬ 
nique  that  uses  the  divide-by-two  approximation  for  the  message  propa¬ 
gation. 


Message  Number 

LAN  Prrtocol  Anal5rzer  Response 
Figure  4C 

Because  this  technique  takes  into  consideration  the  response  time  of 
the  source  to  the  ping  message,  it  is  more  precise  then  the  divide-by-two 
static  message  technique. 
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The  Dynamic  Message  Technique  for  the  clock  offset 

Figure  4D 


There  is  very  little  that  can  be  done  to  improve  the  dynamic  message 
technique.  Attempts  to  refine  the  calculation  usually  result  in  distorting 
the  data  and  introducing  more  unknowns  than  there  were  before  the  re¬ 
finement.  The  problem  gets  complicated  very  quickly. 

To  understand  what  makes  the  approximation  a  rough  approxima¬ 
tion,  a  closer  look  at  the  message  path  is  needed.  In  Figure  4E,  the  mes¬ 
sage  path  is  broken  up  into  its  components,  see  Figure  4E.  Starting  at 
the  observer!  1),  the  time  is  obtained  from  the  clock  and  put  into  the 
message.  The  message  then  goes  from  the  application  in  the  observer 
down  through  the  protocol  layers(2)  across  the  network(3)  up  through  the 
protocol  layers  at  the  source  to  the  application(4)  where  the  time  is  ob¬ 
tained  from  the  clock  on  the  source  clock  and  put  into  the  messagefS). 
The  process  is  repeated  but  this  time  the  message  goes  from  the  source  to 
the  observer.  This  breakdown  has  10  steps  that  the  message  goes 
through  and  each  one  of  them  is  an  unpredictable  delay.  Using  the  dy¬ 
namic  message  technique,  it  is  hard  to  isolate  the  arrival  and  departure 
times  for  each  of  these  steps  without  perturbing  the  measurements. 

The  inability  of  the  dynamic  message  technique  to  account  for  the 
variation  in  the  message  propagation  time  makes  the  technique  less  de¬ 
sirable  then  the  static  message  technique.  The  choice  of  the  technique 
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The  time  required 

1  to  get  and  store  the  departure  time  for  the  message 

2  for  the  message  to  propagate  from  the  observer  application  to  the  network 

3  for  the  message  to  go  from  the  observer  side  to  the  source  side  of  the  network 

4  for  the  message  to  propagate  from  the  network  to  the  source  application 
program 

5  to  get  the  arrival  time  of  the  message  and  put  It  Into  the  message 

6  to  get  the  departure  time  of  the  message  and  put  It  Into  the  message 

7  for  the  message  to  propagate  from  the  source  application  program  to  the 
network 

8  for  the  message  to  go  from  the  source  side  to  the  observer  side  of  the  network 

9  for  the  message  to  propagate  from  the  network  to  the  source  application 
program 

10  to  get  and  store  the  arrival  time  In  the  message 

The  Path  for  Dynamic  Messages 
Figure  4E 

will  depend  on  the  message  generating  capabilities  of  the  system  and  the 
data  resolution  requirements  for  the  analysis.  If  the  system  has  only  a 
static  capability,  then  there  is  only  one  choice.  If  the  system  is  capable  of 
doing  both  the  static  and  dynamic  message  techniques,  then  the  issue  is 
how  precise  does  the  offset  have  to  be.  If  data  requirements  are  in  mil¬ 
liseconds  or  larger,  then  the  dynamic  message  technique  is  preferred  be¬ 
cause  it  requires  fewer  measurements.  If  the  events  are  in  fractions  of  a 
millisecond,  then  the  static  message  technique  is  the  better  choice. 

The  clock  offset  is  a  very  important  part  of  the  global  time  transfor¬ 
mation.  The  techniques  described  in  this  section  are  adequate  for  most 
types  of  analysis.  For  others,  more  precision  is  required  and  those  situa¬ 
tions  require  a  more  elaborate  measurement  scheme  to  improve  the  preci¬ 
sion  of  the  global  time  transformation.  This  effort  didn't  have  enough 
time  or  money  to  research  this  issue. 
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Approach 

In  Sections  2.  3  and  4.  the  global  time  transformation,  clock  rate  and 
the  clock  offset  were  discussed  along  with  the  measurement  techniques  to 
obtain  them.  In  this  section,  those  techniques  were  applied  to  a  real  sys¬ 
tem  and  a  comparison  is  made  between  the  global  time  transformation 
predictions  and  the  measured  clock  values. 

To  do  the  comparison,  two  sets  of  measurements  have  to  be  made. 
The  first  set  is  used  to  determine  the  global  time  transformation.  The 
second  set  is  used  to  measure  clock  drift  (i.e.  via  the  dock  offset)  over  a 
time  interval.  The  dock  rate  from  the  first  set  of  measurements  and  the 
first  dock  offset  from  the  second  set  of  measurements  determine  the  par¬ 
ticular  global  time  transformation  for  this  e^eriment.  The  global  time 
transformation  predictions  of  the  dock  offset  were  made  over  the  same 
time  interval.  These  predictions  and  the  second  set  of  measurements 
were  compared. 

Both  sets  of  measurements  used  the  same  configuration  (see  Figure 
5A).  The  configuration  is  a  simple  two  node  network  with  a  computer  at 
both  ends  of  the  network,  like  the  four  clock  system  discussed  earlier. 
The  two  parallel  computers  are  about  50  miles  apart,  and  are  capable  of 
doing  dynamic  messages.  To  monitor  the  flow  of  the  messages  from  one 
computer  to  the  other  computer,  a  LAN  protocol  Analyzer  (LANPA)  is 
placed  at  each  end  of  the  network.  These  LANPAs  are  not  capable  of  gen¬ 
erating  dynamic  messages. 

Computer  2  Computer  1 

\  Network  t* 


I  ClockY 

LAN  Protocol  Analyzer  2  LAN  Protocol  Analyzer  1 

Verification  Configuration 
Figure  5A 
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Ctecfc.  RaU  Culgulttf  gn 

The  first  set  of  measurements  were  to  determine  the  clock  rate.  The 
three  clock  rates  were  calculated  from  three  separate  groups  of  measure¬ 
ments.  Each  group  of  measurements  contained  a  collection  of  five  exper¬ 
iments  that  were  run  on  one  of  the  systems.  Each  experiment  was  run  as 
long  as  possible  and  was  terminated  whenever  there  was  an  interruption 
in  the  measurement  process.  The  five  experiments  that  made  up  the 
group  ran  for  45,  68.  1 13.  143.  and  233  hours. 

The  function  of  Computer  1  was  to  be  the  observer,  the  controller  for 
the  measurement  process  and  the  reference  clock  (i.e.  the  global  clock)  for 
the  experiment.  The  clock  rate  measurements  were  taken  of  LANPA  1 , 
LANPA  2  and  Computer  2.  Because  the  LANPAs  were  not  capable  of  gen¬ 
erating  dynamic  message,  the  static  message  technique  was  used  for  the 
LANPA  measurements.  Eh^en  though.  Computer  2  was  capable  of  doing 
dynamic  messages,  the  static  message  techniques  were  used  so  that  all 
the  clock  rates  would  be  derived  from  the  same  static  message  technique. 


Expected  lime  (t) 

(Hours) 

A  plot  of  the  233  hour  experiment  for  Computer  2 

Figure  5B 
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One  of  the  plots  for  Computer  2,  (the  233  hours  experiment),  is 
shown  in  Figure  5B.  The  little  boxes  in  the  plot  represent  a  single  mea¬ 
surement. 

The  line  that  goes  through  all  the  boxes  is  the  linear  curve  fit  to  the  data. 
The  equation  for  the  curve  is  also  given  in  the  plot.  The  rest  of  the  exper¬ 
iments  for  Computer  2  and  the  experiments  for  the  two  LANPAs  have 
similar  linear  characteristics  and  are  not  shown. 

The  next  step  is  to  determine  the  clock  rate  for  the  three  systems. 
The  clock  rate  can  be  calculated  by  four  different  methods.  Two  of  the 
methods  used  curve  fitting  techniques.  One  method  uses  a  simple  calcu¬ 
lation  to  approximate  the  clock  rate.  The  last  method  uses  the  equations 
that  were  derived  iii  Section  3. 

In  each  of  the  curve  fitting  methods,  the  curve  fitting  technique  has 
to  be  applied  twice  to  calculate  the  clock  rate.  The  two  curve  fitting 
techniques  used  were  the  least  squares  fitting  and  the  polynomial  fitting 
techniques.  One  of  the  curve  fitting  methods  uses  the  polynomial  fitting 
technique  first  and  the  least  squares  fitting  technique  second  in  the  cal¬ 
culation  of  the  clock  rate.  The  other  curve  fitting  method  used  the  least 
squares  fit  for  both  calculations.  The  only  difference  between  the  two 
methods  for  calculating  the  clock  rate  is  in  the  first  step  where  either  the 
polynomial  fit  or  the  least  squares  fit  techniques  are  applied.  Only  the 
pol3momial  fitting  technique  wiU  be  discussed. 

After  the  data  was  gathered  from  the  experiment,  the  clock  rate  is 
calculated.  The  first  step  is  to  enter  the  data  into  the  polynomial  fitting 
program  and  the  coefiicients  for  the  polynomial  are  calculated  (see  Figure 
5B).  This  is  done  for  aU  five  experiments  in  each  group.  The  coefficients 
are  then  analyzed  to  determine  the  actual  coefficients  for  the  global  time 
transformation. 

Coefflclcnts 

To  determine  the  actual  coefficients,  the  coefficients  are  grouped  by 
term  (i.e.  ao.  al.  a2.  as  and  34)  and  plotted.  Figures  5C  through  5G  are 
the  plots  for  the  groups  of  coefficients  for  Computer  2.^  Figure  5C  is  a 
plot  of  the  ao  coefficient  for  Computer  2.  The  plotted  values  are  the  calcu- 

^  Since  the  three  systems  have  similar  results,  only  the  Computer  2  results  will  be 
discussed. 
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lated  offsets  at  the  beginning  of  the  experiment.  The  measured  values  for 
the  offset  dependent  on  when  the  experiment  is  started  which  is  random. 


ao  Coefficients  for  Computer  2 
Figure  5C 


Duration  (t) 
(Hours) 


ai  Coefficients  for  Computer  2 
Figure  5D 
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Figure  5D  is  the  plot  of  the  ai  coefficients  and  is  the  most  important  plot 
of  all.  The  main  thing  to  notice  is  that  the  difference  between  successive 
Plot  points  gets  smaller  as  the  duration  gets  larger.  This  implies  conver¬ 
gence  as  the  duration  period  gets  larger.  This  is  primarily  due  to  the  fact 
that  the  variations  in  the  message  propagation  become  relatively  small 
when  compared  to  the  duration  time  (see  Equations  15  and  20  in  Section 
3). 

Higher  Order  Coefficients 

Figures  5E.  5F  and  5G  are  the  plots  of  the  a2.  as  and  04  coefficients 
respectively.  Each  of  these  plots  are  similar  in  shape.  They  all  start  at 
high  value  and  then  drop  off  to  a  low  value  as  the  duration  gets  larger. 
From  each  of  the  graphs,  the  value  for  the  coefficient  is  very  small  after 
144  hours  of  duration.  The  sharp  drop  off  for  ai,  a2  and  as  would  imply 
that  experimertts  have  to  be  at  least  144  hours  long  to  provide  a  reasonably 
good  estimate  for  the  coefficient  aj  (le.  the  clock  rate). 

The  differences  in  successive  plot  points  for  the  short  duration  values 
is  large  compared  to  the  differences  for  the  long  duration  values.  The 
large  differences  in  the  shorter  duration  values  when  grouped  in  with  the 
values  for  the  longer  duration  values  introduce  an  error  into  the  calcula¬ 
tion  of  the  ai  coefficient.  The  question  is  how  to  compensate  for  the  large 
differences  in  the  short  duration  values.  Since  the  probability  of  having 
the  correct  ai  is  not  the  same  for  both  the  short  duration  values  and  the 
long  duration  values,  a  weighting  function  can  be  used.  The  weighting 
function  would  make  the  contribution  of  the  short  duration  values  to  the 
calculation  of  ai  smaller  then  the  longer  duration  values.  A  probability 
weighting  function  could  be  applied  to  the  analysis  to  give  a  more  precise 
estimate.  This  approach  was  not  examined  in  this  effort.  The  effects  of 
the  large  variations  in  the  short  duration  values  on  the  calculation  of  ai 
will  be  visible  in  the  comparison  later  in  this  section. 

After  plotting  the  coefficients,  the  rest  of  the  numerical  anal3rsis  is 
performed  to  calculate  the  effective  value  for  the  coefficients.  The  analysis 
is  done  by  applying  the  least  squares  fitting  program  to  each  set  of  coef¬ 
ficient  data.  Like  the  polynomial  fitting  program,  the  least  squares  fitting 
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program  generates  the  coefficients  of  a  line  that  "fits"  the  data.  The 
ficient  for  the  zero  order  term  in  the  equation  for  the  line  is  the  clock 
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Duration 

(Hours) 


a2  Coefficients  for  Computer  2 
Figure  5E 


Duration 

(Hours) 

as  Coefficients  for  Computer  2 
Figure  5F 
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Duration 
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34  Coefficients  for  Computer  2 
Figure  5G 

rate  (see  Figure  5D).  The  first  and  higher  order  terms  for  the  ao  plot  pro¬ 
vide  insight  as  to  how  precise  the  coefficient  value  is  and  the  relative  size 
of  the  error  is  in  the  calculation.  If  the  higher  order  coefficients  are  rela¬ 
tively  small,  then  the  ai  coefficient  is  good  and  the  error  is  small.  The 
higher  order  terms  also  implies  that  the  duration  of  the  experiment 
should  be  greater  than  144  hours. 

To  determine  the  particular  time  transformation  for  this  e:?q)eriment.  a 
clock  offset  is  needed  .  The  clock  offset  for  the  global  time  transformation 
will  come  from  the  first  offset  measurement  in  the  second  set  of  mea¬ 
surements.  Thqr  both  must  have  the  same  initial  clock  offset  if  a  compar¬ 
ison  between  the  global  time  transformation  predictions  and  the  mea¬ 
surement  is  to  be  meaningful. 

Predictions  versus  Measiirements 

With  the  first  set  of  measurements  completed  and  the  global  time 
transformation  defined,  the  next  step  is  to  obtain  the  second  set  of  mea¬ 
surements.  The  second  set  of  measurements  is  a  collection  of  clock  offset 
measurements  that  have  been  taken  on  each  system  over  a  long  interval 
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of  time.  Selecting  the  proper  measurement  technique  is  a  critical  issue  in 
taking  the  measurements.  For  the  experiment,  the  static  message  tech¬ 
nique  was  used  because  the  LANPAs  had  only  a  static  message  capability. 
E>ven  though  Computer  2  has  the  ability  to  do  dynamic  messages,  the 
measurements  for  Computer  2  were  taken  with  the  static  message  tech¬ 
nique.  The  primary  reason  is  that  we  wanted  to  make  sure  that  the 
technique  did  not  have  any  impact  on  the  results.  Therefore,  the  static 
message  technique  was  used  for  both  the  Computer  2  and  both  LANPAs’ 
measurements. 

After  the  second  set  of  measurements  was  taken,  the  initial  clock  off¬ 
set  measurement  was  used  to  define  the  particular  global  time  transfor¬ 
mation.  The  particular  global  time  transformation  for  this  experiment  is 

G(t)  =  0.1984  +  0.061796*  t 

The  clock  offset  measurements  for  Computer  2  and  the  predicted  offsets 
from  the  global  time  transformation  are  shown  in  Figure  5H. 


Duration  (t) 
(Hours) 


Predicted  Offsets  and  Measured  Offsets  for  Computer  2 

Figiu'e  5H 
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The  two  lines  track  each  other  relatively  weU.  To  get  a  better  feel  for 
how  well  they  track  each  other,  let's  look  at  the  results  from  the  other 
three  methods  (see  Figure  51).  The  leeist  squares  fit  method  is  the  same 
as  the  polynomial  fit  method  except  this  method  uses  the  least  squares  fit 
for  the  first  step  of  the  calculation  procedure.  The  resulting  line  is  the 
furthest  from  the  measurements.  Another  method  is  the  extreme  point 
method.  This  method  takes  the  initial  measurements  and  the  final  mea¬ 
surements  and  directly  calculates  the  coefficient.  This  method  produced 
the  best  approximation  to  the  measurements.  This  is  the  same  method 
that  would  be  used  if  the  analyst  had  a  long  time  for  measurements.  The 
last  method  uses  the  equation  derived  in  Section  3  (i.e.  Equations  7.  15. 
and  20)  to  calculate  the  coefficients.  Which  of  these  equations  to  use  de¬ 
pends  on  which  end  of  the  network  the  device  is  on  and  the  device  type. 
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Duration  (t) 
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Comparison  of  the  various  methods 
Figure  51 


5-9 


Verification 


If  the  device  is  on  the  same  side  of  the  network  as  the  global  system,  then 
use  Equation  7.  If  the  device  is  on  the  other  side  of  the  network  and  is  a 
monitoring  device,  then  use  Equation  15.  If  the  device  is  a  computer  sys¬ 
tem  on  the  other  side  of  the  network,  then  use  Equation  20, 

Figure  51  has  some  interesting  points.  The  order  of  the  methods  in 
terms  of  closeness  to  the  measured  clock  offset  is  in  agreement  with  one's 
expectations.  The  extreme  point  method  for  durations  of  this  size  should 
be  the  best  because  the  impact  of  the  variation  in  the  message  propaga¬ 
tion  would  have  the  least  effect  on  the  value.  Since  the  extreme  point 
method  has  no  higher  order  coefficient  values,  the  errors  due  to  them  are 
eliminated.  The  next  best  result  should  be  the  equations  method.  The 
equation  method  is  tailored  to  the  fundamental  problem  because  it  is  de¬ 
rived  from  the  problem  description  (i.e.  the  interconnectivity  model).  The 
curve  fitting  method  would  naturally  be  last  because  it  deals  only  with  the 
data  analysis  and  is  not  tailored  to  the  problem.  The  large  variations  in 
the  data  for  the  short  duration  points  produces  errors  in  calculating  the 
clock  rate  via  the  curve  fitting  method. 

Refinements  to  each  of  the  above  methods,  except  the  extreme  point 
method,  will  generate  a  closer  bunching  of  the  lines.  To  refine  the  curve 
fitting  method,  the  analyst  can  restrict  the  values  to  only  the  longer  dura¬ 
tion  data  points  or  introduce  probability  based  weighting  functions.  The 
equations  method  can  be  enhanced  by  developing  new  approaches  for 
determining  how  to  isolate  and  incorporate  into  the  calculation  additional 
variables  that  effect  the  message  propagation  (e.g.  device  response  times, 
network  utilization  and  adaptive  routing).  New  methods  for  measuring 
the  clock  offset  may  provide  a  better  set  of  coefficients  for  the  global  time 
transformation. 

The  global  time  transformations  that  were  generated  and  compared 
with  the  measured  offsets  were  close.  As  stated  earlier,  an  exact  global 
time  transformation  is  not  possible.  The  goal  of  the  global  time  transfor¬ 
mation  is  to  transform  the  data  with  enough  accuracy  to  satisfy  the  anal¬ 
ysis  requirements. 
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The  global  time  transformation  work  that  was  discussed  in  this  re¬ 
port  came  from  the  information  gathered  during  the  execution  of  an  in- 
house  effort.  Since  the  primary  intention  of  the  in-house  effort  was  to  do 
research  and  not  to  develop  global  time  transformations,  there  is  a  lot  of 
work  that  still  has  to  be  done.  This  section  will  discuss  some  of  the  work. 

The  global  time  transformation  is  a  deterministic  function  that  maps 
the  measured  time  data  into  the  global  time  data.  The  particular  global 
time  transformation  is  obtained  by  calculating  the  coefficients  for  the 
transformation.  To  calculate  the  coefficients  for  the  transformation, 
multiple  experiments  are  run  from  which  multiple  sets  of  coefficient  are 
generated.  From  the  multiple  sets  of  coefficients,  the  global  time 
transformation  coefficients  are  calculated  using  analytical  methods^. 
Since  there  is  a  set  of  candidate  coefficients  to  be  analyzed,  there  is  an 
uncertainty  associated  with  the  coefficient  calculations.  To  effectively 
handle  this  uncertainty,  the  coefficient  should  be  calculated  using  the 
theories  and  anal3rtical  techniques  of  probability.  Since  the  measurement 
environment  is  probabilistic  in  nature,  the  probabilistic  approach  should 
provide  better  values  for  the  coefficients  than  the  deterministic  approach, 
using  probability  to  determine  the  global  time  transformation  is  an  area 
that  requires  additional  research. 

A  different  technique  that  may  improve  the  precision  of  the  coeffi¬ 
cients  calculations  is  to  have  multiple  global  clocks.  With  multiple  global 
clocks,  there  would  be  multiple  sets  of  coefficients  for  calculating  the 
coefficients.  Multiple  coefficients  calculated  by  multiple  observers  would 
enable  the  analyst  to  compare  a  wider  set  of  observations.  The 
coefficients  can  be  numerically  analyzed  not  only  within  the  set  of  values 
of  a  single  observer  but  across  a  set  of  observers.  The  global  time 
transformation  predictions  can  be  analyzed  to  check  for  consistency 
amongst  the  set  of  global  time  transformations,  to  see  if  the  order  of 
events  is  preserved  and  to  miiamize  the  prediction  error  of  the  global  time 


^  In  section  5,  four  of  these  methods  for  calculating  the  coefTicient  were  discussed. 
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transformations.  This  technique  should  enable  the  analyst  to  determine 
more  precisely  the  proper  set  of  global  time  transformations. 

The  experiments  that  were  implemented  during  this  effort  covered 
only  a  few  months.  To  completely  understand  the  characteristics  of  the 
clock  rate,  more  extensive  measurements  and  experimentation  has  to  be 
performed.  The  measurements  and  the  experimentation  should  examine 
the  change  in  the  clock  rate  under  more  rigorously  controUed  conditions. 
The  two  types  of  effects  that  should  be  examined  are  the  physical  and  the 
operational  effects.  The  physical  effects  are  those  changes  in  clock  rate 
due  to  changes  in  the  physical  environment  (e.g.  temperature).  Opera¬ 
tional  effects  are  those  changes  in  the  calculated  clock  rate  due  to 
changes  in  the  operational  environment  (e.g.  system  loading  and  system 
interconnectivity).  The  basic  objective  of  this  research  is  to  determine 
what  and  how  much  does  the  physical  and  operation  environment  impact 
the  calculation  of  the  coefficients  and  the  predictions  of  the  global  time 
transformation. 

The  next  two  recommendations  focus  on  keeping  the  clocks  more 
aligned  with  each  other.  The  first  technique  is  to  use  global  time  trans¬ 
formations  to  predict  clock  offsets  and  then  make  the  system  adjust  the 
clock  rate  to  the  predictions.  The  idea  behind  this  approach  is  to  fix  the 
clock  rate  via  the  global  time  transformation  and  make  the  system  fit  the 
expected  clock  rateor  global  clock  rate.  For  the  global  time 
transformation  not  to  corrupt  the  data,  the  adjustment  process  has  to  be 
uniform  in  time. 

The  other  recommendation  is  to  use  an  expert  system  approach 
which  monitors  NTP  time  messages  and  then  makes  adjustments  to  the 
clock.  The  expert  system  continuously  observes  a  sequence  of  time  mes¬ 
sages  and  learns  from  the  messages  when  and  how  much  the  clock 
should  be  adjusted.  The  expert  system  continues  to  monitor  the  mes¬ 
sages  until  the  expert  system  can  predict  the  occurrence  of  the  messages. 
For  example,  if  someone  tells  you  every  hour  that  your  watch  is  off  by  15 
seconds.  After  a  while  you  can  predict  when  they  will  teU  you  that  your 
watch  is  off  and  by  how  much  it  is  off  (i.e.  15  seconds).  Once  you  know 
when  to  adjust  your  watch  and  by  how  much,  the  adjustment  can  be 
made  regularly  without  the  other  person  telling  you  every  hour. 
Depending  on  the  magnitude  in  the  offset,  the  monitoring  can  be  adjusted 
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to  be  less  frequent  (e.g.  from  every  hour  to  every  day  or  month).  If  the 
adjustment  is  spread  uniformly  over  time,  the  clock  will  be  more  like  a 
perfect  clock  than  those  systems  that  don't  have  this  capability. 

To  supplement  this  global  time  transformation  work,  an  error  analy¬ 
sis  must  be  done.  In  this  effort,  the  error  analysis  was  not  rigorous.  An 
error  analysis  becomes  more  important  when  the  data  resolution  re¬ 
quirements  become  more  demanding.  The  demand  for  more  data  resolu¬ 
tion  occurs  when  the  number  of  events  that  occur  in  a  given  interval  gets 
very  large.  In  today's  concurrent  multiprocessor  environments,  millions 
of  events  occur  in  a  matter  of  seconds.  The  global  time  transformations 
must  be  able  to  transform  the  time  data  well  enough  to  distinguish  the 
events.  The  precision  of  the  transformation  will  d<.pend  on  how  much  the 
numeric  spectrum  is  occupied  by  the  error  bandwidth  in  the  global  time 
transformation.  To  determine  the  resolution  of  the  global  time  transfor¬ 
mation  predictions,  the  analyst  must  determine  the  magnitude  of  the  er¬ 
ror  associated  with  the  calculation  of  the  coefficient  and  with  the  global 
time  transformation  predictions.  For  the  global  time  transformation 
technique  to  become  more  effective,  more  work  has  to  be  done  on  error 
analysis. 
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The  computer  environments  of  today  are  no  longer  contained  in  the 
same  mainframe  or  room  or  even  building.  Today's  computer 
environments  are  multiprocessor  high  performance  computers  that 
interact  with  each  other  via  high  speed  communication  networks.  These 
environments  bring  with  them  not  only  improved  performance  and 
fimctionality  but  also  a  new  breed  of  complex  anal3rtical  problems.  There 
are  many  facets  to  the  problems.  One  of  these  facets  is  that  a  time 
domain  analysis  in  one  of  these  environments  is  a  computational  night¬ 
mare.  The  problem  is  that  the  data  is  generated  from  multiple  systems 
with  different  clocks.  If  the  data  was  generated  from  the  same  clock  the 
problem  would  be  greatly  reduced. 

There  are  two  approaches  to  solving  this  problem,  the  hardware  ap¬ 
proach  and  the  software  approach.  The  hardware  approaches  focus  on 
the  alignment  and  synchronization  of  the  clocks.  These  approaches  are 
either  too  expensive  or  are  not  physically  feasible  because  the  hardware 
uses  chip  technology  and  manufacture  information  is  proprietary.  The 
software  approaches  are  better  because  they  don't  have  to  deal  with  the 
physical  restrictions  of  the  environment  but  deal  only  with  the  data, 
which  makes  these  approaches  less  costly  and  easier  to  implement. 

The  global  time  transformation  technique  is  one  of  those  software  ap¬ 
proaches.  The  basic  premises  of  the  global  time  transformation  approach 
are  that  there  are  no  perfect  clocks  and  that  the  time  data  comes  from 
clocks  that  are  out  of  alignment  and  out  of  synchronization.  The  global 
time  transformation  is  a  very  simple  technique  that  provides  very  good 
results.  Fortunately,  the  transformation  is  a  linear  ftmction  which  makes 
data  transformation  significantly  easy  to  implement. 

Since  it  is  not  possible  to  correct  the  underlying  causes  of  the 
multiple  clock  system  problem,  the  global  time  transformation  corrects 
the  effects  of  the  problem.  Approaches  that  deal  with  the  effects 
generated  by  multiple  clock  systems  will  have  a  higher  success  rate  then 
those  that  try  to  correct  the  cause  of  the  problem.  The  global  time 
transformation  deals  only  with  the  effects  of  the  problem.  The  work 
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discussed  in  this,  report  only  deals  with  some  of  the  techniques  for 
generating  a  global  time  transformation.  There  is  room  for  new 
techniques  to  be  developed  and  room  for  improvement  to  the  techniques 
mentioned  in  this  report. 

This  report  has  shown  that  global  time  transformations  are  linear 
and  quite  precise.  To  generate  the  global  time  transformation,  an  analyst 
has  techniques  available  to  him/her  that  can  be  used  to  measure  data 
and  to  calculate  the  transformation  coefficients.  When  the  coefficients  are 
substituted  into  the  global  time  transformation,  the  analyst  will  be  free  to 
analyze  multiple  clock  S3rstems  in  the  same  way  as  he/she  analyzed  single 
clock  systems. 

The  global  time  transformation  is  a  candidate  solution  to  the  analyst’s 
multiple  clock  data  problem. 
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MISSION 

OF 

ROME  LABORATORY 


Mission.  The  mission  of  Rome  Laboratory  is  to  advance  the  science  and 
technologies  of  command,  control,  communications  and  intelligence  and  to 
transition  them  into  systems  to  meet  customer  needs.  To  achieve  this, 
Rome  Lab: 


a.  Conducts  vigorous  research,  development  arKi  test  programs  in  ail 
applicable  technologies; 

b.  Transitions  technology  tc  current  and  ^ure  systems  to  improve 
operational  capability,  readiness,  and  supportability; 

c.  Provides  a  full  range  of  technical  support  to  Air  Force  Materiel 
Command  product  centers  and  other  Air  Force  organizations; 

d.  Promotes  transfer  of  technology  to  the  private  sector; 

e.  Maintains  leading  edge  technological  expertise  in  the  areas  of 
surveillance,  communications,  command  and  control,  intelligence,  reliability 
science,  electro-magnetic  technology,  photonics,  signal  processing,  and 
computational  science. 


The  thrust  areas  of  technical  competence  include:  Surveillance, 
Communications,  Command  and  Control.  Intelligence,  Signal  Processing, 
Computer  Science  and  Technology,  Electromagnetic  Technology. 
Photonics  and  Reliability  Sciences. 


